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EXECUTIVE  SUMMARY 


Poor  maintenance  is  the  foremost  cause  of  operational  readiness 
shortfalls  in  today's  Navy,  and  inadequate  skills  are  the  root-cause.  The  gap 
between  required  and  available  maintenance  skills  is  explained,  in  part,  by  a 
shortage  of  senior  petty  officers  in  combat  systems  maintenance  ratings.  The 
primary  factor,  however,  is  inadequate  technical  training.  This  is  evidenced 
by  an  increasing  dependence  of  the  Fleet  on  nonorganic  technical  assistance 
for  casualty  repairs.  Even  more  overwhelming  evidence  is  provided  by 
indicators  of  maintenance  skill  deficiencies,  such  as  excessive  no-fault 
removals . 

Case  studies  of  several  combat  subsystems  confirm  that  the  Navy's  formal 
training  approach  does  not  provide  a  sufficient  foundation  of  knowledge  and 
skill  for  technicians  to  develop  maintenance  job  proficiency  through 
experience.  In  essence,  the  training  graduate  entering  the  Fleet  today  is  a 
novice,  not  an  apprentice-level  technician  as  was  the  case  ten  years  ago. 

The  Navy  is  caught  in  a  vicious  cycle  of  declining  self-sufficiency  and 
skill  stagnation.  Simple  repairs  must  be  done  by  senior  petty  officers  ( E— 5 
and  E-6 )  because  the  junior  petty  officers  (E-4)  do  not  possess  the  needed 
maintenance  skills.  Shipboard  manning  is  not  designed  for  this  substitution, 
consequently,  no  skilled  man-hours  are  left  to  cope  with  complex  repairs. 
Complex  repairs,  therefore,  must  be  done  with  nonorganic  technical 
assistance.  As  a  consequence,  skilled  technicians  stagnate  at  whatever  skill 
level  they  possess,  while  apprentice  technicians  do  not  grow  in  proficiency  at 
a  rate  required  to  attain  journeyman-level  skills  during  their  first  terms. 
This  cycle  feeds  on  itself  with  the  consequences  becoming  increasingly 
pronounced.  The  cycle  must  be  stopped  if  the  Navy  is  to  continue  meeting  its 
mission  requirements.  The  only  way  to  turn  the  situation  around  is  through 
significant  improvements  in  technical  training. 

Several  critical  issues  in  maintenance  training  and  aiding  technology 
should  be  addressed  by  research  and  development  (R&D).  For  example,  the 
present  knowledge  about  cost-effective  training  of  cognitive  skills  (such  as 
troubleshooting)  is  embryonic.  Acquisition  and  use  of  effective  maintenance 
training  equipment  or  training  delivery  systems  suffer  from  a  lack  of 
knowledge  about  the  interactions  among  design  characteristics,  training 
objectives,  training  sequence,  and  individual  differences.  Implementation  of 
job-aiding  technology  is  limited  by  incomplete  understanding  of  what  technical 
information  in  what  format  is  needed  by  the  technician  and  how  tutorial 
software  should  be  structured  to  maximize  on-the-job  skill  acquisition. 

Based  upon  the  results  of  this  study,  it  is  recommended  that  the  Navy: 

Adopt  a  coordinated  R&D  program  in  maintenance  training  and  aiding 
technology  with  the  single  objective  of  ensuring  that  maintenance 
technicians  acquire  and  maintain  the  knowledge  and  skills  required 
for  proficient  job  performance. 
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Give  high  priority  to  projects  on  the  following  subjects: 

technical  information  needs  in  maintenance  performance, 
expert  troubleshooting  strategies, 
specifications  and  guides  for  job-aiding  software, 
a  Fleet  training  delivery  system, 

expanded  task-analysis  procedures  to  fill  the  voids  in  current 
instructional  system  development  (ISD)  guides, 

relationships  among  training  equipment  design  characteristics, 
transfer-of-training,  and  training  objectives, 

a  decision  support  system  (training  equipment  acquisition 
manual  and  supporting  data  base)  to  assist  Systems  Command 
program  managers  and  Office  of  the  Chief  of  Naval  Operations 
sponsors  in  training  equipment  acquisition, 

specifications  for  general-purpose  and  generalized  maintenance 
training  equipment,  including  standardized  modules  for  student 
and  instructor  stations. 

These  R&D  efforts  will  be  necessary,  but  not  sufficient.  Significant 
improvements  in  maintenance  skills  require  a  much  broader  program,  addressing 
other  key  factors  influencing  job  performance:  occupational  structure,  job 
design,  training  philosophy,  instructional  systems  development,  individual 
performance  monitoring,  personnel  management,  human  factors  engineering,  and 
design  for  maintainability.  Skill-deficiency  is  a  systemic  problem  influenced 
by  numerous  interrelated  factors.  Better  maintenance  training  equipment  and 
job  aids  will  pay  off  in  improved  job  performance  only  if  comparable 
improvements  are  made  in  the  other  factors.  The  Navy  has  several  R&D  efforts 
underway  to  foster  improvements  in  selected  areas.  Included  are  the  Enlisted 
Personnel  Individualized  Career  System,  the  Navy  Technical  Information 
Presentation  System,  the  Design  for  Maintainability  Program,  and  the  Testing 
Technology  Program.  Uhat  appears  to  be  lacking  is  a  coordinated  plan  of 
action  addressing  all  of  the  chief  contributors  to  poor  maintenance 
performance. 

On  a  more  global  basis,  it  is  recommended  that  the  Navy: 

Institute  a  comprehensive  Maintenance  Improvement  Plan  for 
management  action  on  the  chief  contributors  to  poor  maintenance 
performance. 

Appoint  a  task  force  to  review  and  manage  execution  of  the 
Maintenance  Improvement  Plan. 
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Create  a  permanent  office  in  Headquarters,  Naval  Material  Command 
responsible  for  monitoring  technology  trends,  assessing  their 
impacts  on  maintenance  shill  requirements,  and  disseminating  that 
information  among  the  manpower,  personnel,  training,  and  logistics 
support  communities. 
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PREFACE 


This  study,  which  was  performed  during  the  1982-1984  time  frame,  was 
co-sponsored  by  the  Naval  Training  Equipment  Center  (Human  Factors  Division) 
and  the  Navy  Personnel  Research  and  Development  Center  (Training  Research 
Directorate)  by  direction  of  the  Deputy  Chief  of  Naval  Operations  (Manpower, 
Personnel,  and  Training),  Research,  Development  and  Studies  Branch  (OP-01B7, 
formerly  OP-115).  The  purpose  of  the  study  was  to:  (1)  develop  a  descriptive 
model  of  Navy  maintenance,  defining  and  assessing  known  factors  which 
influence  maintenance  performance,  and  (2)  develop,  on  the  basis  of  this 
model,  a  systematic  research  and  development  (R&D)  program  in  maintenance 
training  and  aiding,  responsive  to  current  and  projected  needs,  to  improve 
maintenance  skills  and  performance.  This  Final  Report  presents  the  study 
findings  and  conclusions  and  describes  the  recommended  R&D  program. 

In  the  conduct  of  this  study,  Logistics  Management  Institute  (LMI)  was 
supported  by  the  following  subcontractors  in  the  subject  areas  indicated: 
Applied  Science  Associates,  Inc.  (Dr.  Andrew  P.  Chenzoff  and  Hr.  Reid  P. 
Joyce):  technical  data  and  maintenance  job  aids;  Parker  Consultant 
Associates,  Inc.  (Mr.  Edward  L.  Parker  and  Mr.  Richard  I.  Merrell): 
maintenance  training  equipment;  and  Advanced  Research  Resources  Organization 
(Dr.  Edwin  A.  Fleishman  and  associates):  individual  abilities. 

In  the  formulation  of  specific  R&D  recommendations,  LMI  benefitted  from 
the  experience,  ideas,  and  comments  of  many  individuals  beyond  the 
subcontractors  listed  above.  While  LMI  assumes  responsibility  for  the  final 
product,  we  gratefully  acknowledge  the  following  contributors:  Dr.  Gerard  M. 
Deignan  (Air  Force  Human  Resources  Laboratory),  Dr.  Ralph  DePaul,  Jr.  (DETEX 
Systems,  Inc.),  Mr.  Daniel  J.  Dwyer  (Naval  Training  Equipment  Center), 

Dr.  John  D.  Fletcher  (University  of  Oregon),  Dr.  John  D.  Folley  (Applied 
Science  Associates,  Inc.),  Dr.  Kay  Inaba  (XYZYX  Information  Corporation), 

Dr.  William  E.  Montague  (Navy  Personnel  Research  and  Development  Center), 

Mr.  William  A.  Rizzo  (Naval  Training  Equipment  Center),  Dr.  Robert  J.  Seidel 
(Army  Research  Institute  for  the  Behavioral  and  Social  Sciences),  and  Dr. 
Wallace  H.  Wulfeck,  II  (Navy  Personnel  Research  and  Development  Center). 
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individual  skill  proficiency,  and  the  quality  of  job  aids.  Individual 
proficiency  is  a  function  of  experience,  training,  and  individual  abilities. 
"I"-level  maintenance  performance  is  influenced  by  the  same  factors  and, 
additionally,  by  the  availability  of  automated  test  equipment  (ATE)  and  the 
quality  of  test  program  sets  (TPS)  for  checkout  and  fault  isolation  of 
components  and  modules  from  the  prime  equipment.  "D"-level  maintenance  is 
classified  as  indirect  maintenance  and  is  not  of  interest  to  this  study. 

Figure  1  (page  10)  shows  only  the  primary  interactions  between  the 
various  factors  (ultimately,  all  factors  are  interrelated),  and  identifies 
those  interactions  which  are  the  weak  links  in  weapon  system  acquisition  and 
support.  Strengthening  those  links  would  alleviate  much  of  the  maintenance 
performance  problem. 

SUMMARY  OF  CASE  STUDIES 

The  maintenance  support  of  several  combat  systems/subsystems  was  examined 
during  this  study.  Systems/subsystems  from  the  surface,  air,  and  subsurface 
communities  were  included  in  the  sample.  The  findings  and  observations,  which 
are  based  on  information  collected  through  Fleet  visits,  3-M  data,  and 
previous  Navy  and  contractor  studies,  are  summarized  in  the  following 
subsections . 

READINESS  SHORTFALLS.  Any  discussion  of  readiness  shortfalls  is  open  to 
arguments  about  the  measurement  of  readiness,  specification  of  goals,  and  true 
mission  capabilities.  The  subject  of  readiness  measurement  is  complex;  many 
studies  have  been  conducted  on  improving  current  readiness  indicators  as  well 
as  readiness  management  (relating  resource  inputs  to  valid  output  measures  of 
readiness).  This  study  is  not  intended  to  address  these  issues,  but  cannot 
avoid  commenting  on  current  measures  in  order  to  clarify  what  is  meant  by 
"readiness  shortfalls."  Arguments  about  the  adequacy  or  inadequacy  of  current 
measures  and  goals  for  material  readiness  are  inspired  essentially  by  the 
following  three  points: 

a.  Multimission  platforms  with  combat  systems  possessing  multi- 
operational  modes  are  acquired  exactly  for  that  purpose:  to  be  capable  of 
performing  multiple  missions  exploiting  many  different  operational  modes. 

Yet,  the  Navy's  formal  goals  (and,  therefore,  the  readiness  measures  reported) 
are  stated  in  terms  of  selected  (primary)  missions  and  require  only  selected 
(mostly  one)  operational  modes  to  be  "up." 

b.  Reaching  for  stated  MC  goals,  which  invariably  happens  in  the  Fleet, 
may  conflict  with  goals  that  are  more  meaningful  for  the  Fleet.  For  example, 
in  the  case  of  Naval  aircraft,  it  may  be  operationally  more  desirable  to  have 
50  percent  of  the  squadron  in  FMC  status  and  50  percent  in  NHC  status  than 
achieving  the  stated  MC  goal  with  all  aircraft  in  PMC  status. 

c.  The  criteria  applied  in  operational  tests  and  acceptance  tests 
generally  reflect  the  true  operational  requirement;  i.e.,  the  reason  why  the 
weapon  system  was  developed.  Yet,  the  criteria  applied  in  readiness  reporting 
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TABLE  1.  STANDARD  READINESS  MEASURES 


Cl  (fully  ready):  A  unit  fully  capable  of  performing  the  biisioo  for 
which  it  is  organized  or  designed. 

UNIT 

READINESS 

C2  (substantially  ready):  A  unit  has  minor  deficiencies  which  limit 
its  capability  to  accomplish  the  mission  for  which  it  is  organized  or 
designed . 

C3  (marginally  ready):  A  unit  has  major  deficiencies  of  such  magni¬ 
tude  as  to  limit  severe*/  its  capability  to  accomplish  the  mission 
for  which  it  is  organized  or  designed. 

CU  (not  ready):  A  unit  not  capable  of  performing  the  mission  for 
which  it  ia  organized  or  designed. 

C5:  Applies  to  weapon  systems  in  overhaul. 

FMC  (fully  misaion  capable):  When  all  mission-essential  subsystems 
ire  installed  and  operating  as  designated  by  the  Service. 

MATERIAL 

CONDITION 

PMC  (partial  misaion  capable):  When  systems  can  perform  one  or  more 
but  not  all  assigned  missions  because  one  or  more  of  their  j 
mission-essential  subsyitems  are  inoperative  for  maintenance  or 
supply  reasons. 

MC  (mission  capable):  The  suamation  of  FMC  and  PMC. 

KMC  (not  mission  capable):  When  none  of  the  assigned  missions  can  be 
performed  either  due  to  maintenance  (KM CM)  or  supply  (NMC5). 

.  _  Uptime  _  MTBM 

o  Uptime  ♦  Downtime  MTBM  ♦  MDT  * 

OPERATIONAL 

AVAILABILITY 

MTBM  =  mean  time  between  maintenance 

MDT  v  mean  down  time  =  MTTR  ♦  HIST  ♦  HAST 

MLDT  =  mean  logistics  delay  time  (awaiting  parts) 

NADT  -  mean  administrative  delay  time 

MI  IK  v  Man  time  to  repair. 

SUMMARY  DESCRIPTION  OF  THE  MODEL.  By  virtue  of  its  definition,  A0  (see 
Figure  1)  is  determined  directly  by  equipment  reliability,  spares  availability 
or  responsiveness  of  the  supply  system,  availability  of  tools  and  operable 
(calibrated)  test  equipment.  Fleet  support,  and  organizational  (M0"-level) 
maintenance  performance.  The  higher  levels  of  maintenance,  intermediate 
(“I" -level)  and  depot  ("D"-level),  interact  with  the  H0"-level  both  directly 
and  indirectly.  Directly,  poor  "0"-level  maintenance  overburdens  the  higher 
levels  and  vice  versa.  Indirectly,  the  various  maintenance  levels  interact 
through  test  equipment  and  spares  availability  because  "I"-  and/or  ”D"-level 
maintenance  determine  the  turnaround  time  for  test  equipment  repair/ 
calibration  and  module  repairs.  "0"-level  maintenance  performance  is  further 
influenced  by  equipment  characteristics  (reliability,  maintainability,  and 
maintenance  concept),  operational  environment,  organization  and  job  design. 
Fleet  practices,  and  maintenance  capability  (available  skills).  The  latter, 
in  turn,  is  determined  by  the  availability  of  maintenance  personnel,  their 
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READINESS  MEASURES.  Maintenance  is  not  an  end  in  itself.  The  sole  purpose  of 
maintenance  is  to  achieve  and  sustain  combat  readiness;  i.e.,  the  ability  of  a 
unit  to  perform  the  missions  or  functions  assigned.  Combat  readiness  is  the 
composite  of  personnel  readiness  (determined  by  personnel  availability  and 
training)  and  materiel  readiness  (determined  by  logistics  support).  A  measure 
of  combat  readiness  is  provided  by  the  "C"  ratings  of  the  unit  status  and 
identity  reporting  (UNITREP)  system,  a  standard  Department  of  Defense 
(DoD)-wide  reporting  system  instituted  by  the  Joint  Chiefs  of  Staff  (JCS). 
UNITREP  combines  resource-specific  (personnel,  supply,  equipment,  and 
training)  and  mission-area  (antisubmarine  warfare,  antisurface  warfare, 
antiair  warfare,  and  mobility)  ratings  in  reporting  an  overall  readiness  ("C") 
rating  in  terms  of  five  levels  of  readiness  (see  Table  1). 

Material  readiness  is  also  measured  and  reported  separately  through 
different  systems.  One  is  the  DoD  Materiel  Condition  Reporting  System  which 
measures  material  readiness  in  terms  of  mission  capable  (MC)  rates;  i.e.,  the 
percentage  of  time  (hours)  spent  in  each  of  the  three  material  conditions: 
fully  mission  capable  (FMC),  partially  mission  capable  (PMC),  and  not  mission 
capable  (NMC)  (see  Table  l)1*.  (For  Army  and  Marine  Corps  ground  equipment, 
time  is  measured  in  terms  of  days,  not  hours,  such  that  downtimes  of  less  than 
12  hours  duration  are  not  counted.)  MC  reporting  in  the  Navy  has  been 
implemented,  so  far,  only  for  naval  aviation.  Another  measure  is  operational 
availability  (A0);  i.e.,  the  probability  that  equipment  will  be  ready  for 
use  when  needed  at  any  given  time  (in  the  operational  environment).  A0  can 
be  measured  in  different  ways;  the  standard  equation  is  shown  in  Table  1.  The 
MC  rate  is  primarily  an  operational  (and  historical)  measure  of  material 
readiness,  while  A0  is  a  logistics  (and,  perhaps,  more  predictive)  measure 
of  material  readiness.  The  two  differ  as  a  result  of  different  definitions 
and  accounting  rules.  Limited  operating  hours  tend  to  inflate  MC  rates  but 
have  less  impact  on  A0.  In  general,  the  computed  A0  for  a  weapon  system 
is  less  than  the  reported  MC  rate.  Furthermore,  the  norm  in  the  Navy  is  to 
compute  the  A0  for  multimission  systems  separately  by  mission  area. 

In  sum,  readiness  or  availability  data  must  be  interpreted  with  great 
caution,  giving  due  consideration  to  the  definitions  applied.  The  model  uses 
A0  as  the  output  measure  of  the  maintenance  system. 


U-DoD  Instruction  7730.25,  "Materiel  Condition  Reporting  For  Mission- 
Essential  Systems  and  Equipment,"  May  1980.  The  MC  rates  defined  in  this 
instruction  replace  the  operational  readiness  rates  defined  in  the  1972 
version  of  the  same  instruction.  The  revised  instruction  requires  each 
Service  to:  (1)  establish  MC  goals,  (2)  monitor  the  rates  actually  achieved, 
(3)  identify  the  top  five  problems  at  the  subsystem  level  causing 
nonachievement  of  goals,  and  (A)  develop  remedial  actions  necessary  to  correct 
those  problems. 
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CRITERION  MEASURES.  Maintenance  performance  can  be  measured  in  many  different 
ways:  on  an  individual,  team,  or  unit  basis.  The  model  uses  the  following 
four  measures  on  an  aggregate,  unit  basis: 

a.  Capability.  Measured  in  terms  of  the  percent  of  maintenance  tasks 
accomplished  without  outside,  nonorganic  assistance. 

b.  Proficiency.  Measured  in  terms  of  the  repair  time  (including 
diagnostic  time  but  excluding  parts  awaiting  or  logistics  delay  time)  observed 
(denominator)  compared  to  that  required  by  an  expert  technician  (numerator). 

c.  Quality.  Measured  in  terms  of  three  benchmarked  scales  capturing 
the  following  dimensions  of  quality  of  maintenance  performance. 

(1)  Effectiveness.  Extent  to  which  maintenance  returns  the 
equipment  to  fully  operational  performance  capability. 

(2)  Efficiency.  Extent  to  which  good  modules  are  erroneously 
removed  and  replaced  (NFR  rate)  as  evidenced  by  retest  okay  data  at  the 
intermediate  or  depot  levels  of  maintenance. 

(3)  Reliability.  Extent  to  which  maintenance-induced  damage  is 
inflicted  on  the  equipment  or  the  reliability  of  the  equipment  is  degraded 
following  maintenance  action. 

d.  Sustainability.  Measured  in  terms  of  three  benchmarked  scales 
capturing  the  following  dimensions  of  wartime  sustainability. 

(1)  Self-sufficiency.  Extent  of  capability  to  perform  emergency 
repairs  with  no  outside  support,  including  repairs  routinely  allocated  to 
higher  maintenance  levels. 

(2)  Battle  Damage  Repair.  Extent  of  capability  to  return  battle 
damaged  equipment  to  operation,  even  if  degraded. 

(3)  Endurance.  Extent  to  which  battle  conditions  (intensity  and 
duration)  can  be  endured. 

These  measures  are  theoretical  constructs  which  cannot  be  quantified  from 
the  maintenance  data  routinely  collected  by  the  Navy  (the  Maintenance  and 
Material  Management  (3-M)  System,  Casualty  Reporting  (CASREP)  System,  and 
technical  assistance  data  from  MOTUs  and  contractors).  As  a  result,  the  model 
can  only  be  descriptive,  not  quantitative.  The  case  studies  were  used  to 
establish  rough  orders  of  magnitude  of  maintenance  performance  and  to  rank 
order  the  various  factors  influencing  maintenance  performance  in  terms  of  the 
first  three  measures.  The  fourth,  and  perhaps  the  most  important  criterion 
measure,  is  influenced  by  factors  beyond  the  scope  of  this  study  (e.g., 
wartime  scenarios,  Naval  Reserve  issues,  etc.). 
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SECTION  II 

COMBAT  SYSTEMS  MAINTENANCE  PERFORMANCE 


FACTORS  AND  INTERACTIONS 

The  factors  influencing  maintenance  performance  and  the  interactions 
between  these  factors  are  illustrated  in  Figure  1.  A  summary  description  of 
the  model  is  included  here  in  order  to  explain  the  study  approach  and  the 
terminology  used  in  this  report. 


Figure  1.  Simplified  Model  of  Maintenance  Performance 
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c.  Recommendation  #3.  Create  a  permanent  office  in  Headquarters,  Naval 
Material  Command,  responsible  for  monitoring  technology  trends,  assessing 
their  impacts  on  maintenance  skill  requirements,  and  disseminating  this 
information  among  the  manpower,  personnel,  training,  and  logistics  support 
communities. 

d.  Recommendation  //4.  Revise  the  Navy  Training  Improvement  Program  to 
include  the  specific  R&D  projects  in  maintenance  training  and  aiding 
technology  identified  in  this  report. 
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factors  influencing  maintenance  performance,  e.g.,  inadequacies  in  training 
philosophy,  job-task  analysis,  job  design,  and  instructional  systems 
development.  These  factors  appear  to  be  beyond  the  specific  research  issues 
addressed  by  NAVTIP.  The  effective  date  of  NAVTIP,  originally  planned  for 
October  1983,  was  deferred  by  lack  of  funding. 

ONLY  A  BALANCED  AND  COMPREHENSIVE  PROGRAM  CAN  OFFER  A  SOLUTION  TO  THE 
PROBLEM.  The  data  provided  in  this  report  demonstrate  that  maintenance 
deficiencies  are  now  reaching  critical  proportions.  The  problem  is  systemic, 
yet  the  Navy  has,  time  and  again,  attempted  to  solve  the  problem  through 
piecemeal  actions.  In  our  opinion,  there  has  been  little  noticeable 
improvement  in  maintenance  performance;  we  believe  the  same  may  also  be  true 
for  NAVTIP  because  of  its  exclusive  focus  on  training  and  aiding  technology. 
What  is  needed  is  a  broad  and  balanced  program  addressing  all  key  factors 
influencing  maintenance  performance.  Section  IV  describes  how  current  job 
design,  organization  design,  maintenance  philosophy,  training  philosophy, 
instructional  system  development,  personnel  utilization  and  classification, 
maintenance  data,  technical  manuals,  and  supply  support  all  play  a  key  role 
which  may  inhibit  achieving  the  improvements  in  maintenance  performance 
expected  from  such  initiatives  as  design  for  maintainability,  NAVTIP,  and 
advanced-technology,  maintenance  training  equipment  and  aids.  Until  and 
unless  the  Navy  is  committed  to  tackle  the  problem  in  all  its  dimensions,  the 
situation  will  probably  deteriorate  further.  The  short-term  impacts  are 
susceptible  to  compensation  through  unusual  and  expensive  efforts  by  the 
Fleet;  the  long-term  consequences,  however,  bode  ill  for  the  future  support  of 
more  complex  combat  systems. 

RECOMMENDATIONS 

The  present  study  has  attempted  to  consolidate  the  current  state-of- 
knowledge  with  regard  to  Navy  combat  systems  maintenance  today  and  in  the 
mid-1990s;  the  factors  influencing  maintenance  performance;  the  cause-effect 
relationships  which  are  key  to  today's  performance  shortfalls;  and  the  voids 
in  knowledge  requiring  R&D  efforts  to  improve  maintenance.  Undoubtedly,  the 
study  is  incomplete;  however,  it  may  serve  as  a  start  for  a  Navy-wide  effort 
to  improve  combat  system  maintenance.  The  Navy  possesses  considerable 
in-house  R&D  capabilities  which  should  be  more  fully  utilized  through  a 
coordinated  and  focused  plan  in  support  of  the  needs  of  the  Fleet.  This  study 
provides  the  blueprint  for  such  a  plan.  The  study  also  identifies  the  impact 
future  technology  may  have  on  maintenance  skill  requirements.  Section  V 
present s  recommendations  which  are  summarized  as  follows: 

a.  Recommendation  #1.  Institute  a  comprehensive  Combat  System 
Maintenance  Improvement  Program  for  management  action  on  the  chief 
contributors  to  poor  maintenance  performance. 

b.  Recommendation  //2.  Appoint  a  task  force  to  review  and  manage 
execution  of  the  program;  use  the  results  of  the  present  study  in  formulating 
the  Plan  of  Action  and  Milestones  to  execute  the  program. 
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perform  the  routine  maintenance  tasks  expected,  so  the  E-5  and  E-6  must  do 
these  tasks  instead.  Due  to  the  shortage  of  skilled  E-5/E-6  technicians,10 
these  experienced  technicians  are  preoccupied  with  routine  maintenance  tasks 
and  need  help  for  the  less-frequent  complex  tasks,  and  chief  petty  officers 
have  become  managers  too  busy  with  paperwork.  The  end  result  is  that  outside 
assistance  is  necessary  and  skills  are  stagnating  at  all  levels.  It  is  only 
very  recently  that  one  Type  Commander,  Naval  Surface  Force,  U.S.  Atlantic 
Fleet  (COMNAVSURFLANT) ,  has  recognized  that  top-level  action  is  necessary  to 
break  this  cycle  of  skill-stagnation.  The  resulting  Combat  Systems 
Improvement  Program  (CSIP)  is  focused  on  meeting  technical  training 
shortfalls,  but  this  program  is  still  in  an  embryonic  stage. 

THE  NAVY'S  APPROACH  TO  THE  PROBLEM  HAS  BEEN  LARGELY  ONE-SIDED.  There  are  two 
sides  to  the  problem:  a  demand  side  (skills  and  knowledge  requirements  as 
determined  by  system  design  characteristics  and  technical  manuals  (TM)  or  aids 
available  to  the  maintainers)  and  a  supply  side  (skills  and  knowledge 
possessed  by  technicians  as  determined  by  training,  experience,  and  individual 
abilities).  To  close  the  existing  gap  between  the  two,  both  offer 
opportunities  for  improvement.  The  Navy,  so  far,  has  focused  its  initiatives 
on  the  demand  side  of  the  problem  and  has  chosen  to  downplay  the  supply  side. 
Programs  such  as  design  for  maintainability  and  design  for  testability  have 
resulted  in  new  standards,  specifications,  and  design  guides  for  use  in 
acquisition  of  new  weapon  systems.  If  enforced,  one  might  expect  future 
reduction  in  maintenance  task  difficulty  compared  to  what  otherwise  might  have 
resulted  (not  necessarily  a  reduction  compared  to  today’s  maintenance  tasks  as 
a  result  of  increasing  complexity).  Similarly,  the  Navy  Technical  Information 
Presentation  System,  if  implemented,  would  improve  the  quality  of  technical 
information  available  to  the  maintainer.  Both  types  of  programs  will  permit 
an  improvement  in  maintenance  performance. 

On  the  supply  side,  however,  similar  initiatives  have  been  lacking. 

While  the  Navy  has  spent  large  amounts  on  R&D  in  personnel  and  training,  it 
has  been  largely  unable  or  unwilling  to  transfer  results  into  use.  This 
phenomenon  was,  once  again,  noted  by  the  Defense  Science  Board  (DSB)  in  the 
1982  Summer  Study  on  Training  and  Training  Technology  (see  Appendix  B  for  a 
summary  of  this  study  and  Appendix  C  for  the  decision  memorandum  from  the 
Secretary  of  Defense  approving  the  DSB  recommendations).  A  number  of 
circumstances  (increasing  complaints  by  the  Fleet,  the  DSB  1982  Summer  Study, 
and  the  Chief  of  Naval  Research's  urging  that  the  Navy  utilize  results  from 
past  R&D  investments)  have  led  to  the  recent  creation  of  the  Navy  Training 
Improvement  Program  (NAVTIP).  Appendix  D  describes  how  this  program  is 
currently  structured.  NAVTIP  represents  the  first  Navy-wide  program 
addressing  the  supply  side  of  the  maintenance  problem.  NAVTIP  points  in  the 
right  direction  but  still  leaves  unaddressed  some  key,  training-related 


1°C  aspar  W.  Weinberger  (Chairman),  Military  Manpower  Task  Force  Report 
to  the  President  on  the  Status  ar.d  Prospects  of  the  All  Volunteer  Force, 
Washington,  DC,  October  1982. 
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designed  as  an  emergency  program  to  fill  voids  in  organic  maintenance 
capabilities,  it  has  become  a  permanent  fixture  in  today's  Navy. 

Third,  shill  deficiencies  resulting  in  inefficient  troubleshooting  and 
high  no-fault  removal  (NFR)  rates  have  little  impact  on  combat  system  A0  as 
long  as  the  supply  system  supports  this  with  adequate  stockage  of  spares.  In 
this  sense,  the  supply  system,  if  funded  well,  can  compensate  for  poor 
maintenance  performance.  Problems  arise  only  when  funding  limitations  and/or 
long  turnaround  times  for  depot-level  repairs  limit  this  option,  as  happened 
in  the  second  half  of  the  1970s.  Available  data  indicate  that  the  Fleet  is 
relying  heavily  on  the  supply  system's  ability  to  compensate  for  poor 
maintenance:  depending  on  combat  system,  30  to  SO  percent  of  weapon 
replaceable  assemblies  and  60  to  80  percent  of  printed  circuit  boards  returned 
to  higher  maintenance  levels  for  checkout  and  repair  are  found  ready  for 
issue;  i.e.,  were  unnecessarily  removed  from  the  combat  system.9  While  this 
high  NFR  rate  has  several  contributing  causes  (e.g.,  maintenance  concept, 
tolerance  incompatibility  between  the  combat  system  and  test  program  sets 
(TPS),  operational  environment),  lack  of  troubleshooting  skills  is  the  chief 
cause.  With  the  steep  cost  increases  of  spare  parts  experienced  with 
successive  generations  of  increasingly  complex  combat  systems,  the 
affordability  of  this  option  to  compensate  for  skill  deficiencies  is  becoming 
less  and  less. 

In  sum,  the  costs  incurred  by  the  Navy,  to  compensate  somehow  for 
maintenance  deficiencies,  is  impossible  to  determine  accurately.  A 
conservative  estimate  is  that  it  runs  into  several  billions  of  dollars 
annually,  just  for  combat  systems. 

LEAVING  THE  PROBLEM  UNSOLVED  HAS  STARTED  THE  NAVY  ON  A  VICIOUS  CYCLE  OF  COMBAT 
SYSTEM  MAINTENANCE  SKILL  STAGNATION  AND  DEGRADATION.  Because  the  Fleet  must 
meet  its  mission  requirements  in  peacetime,  it  has  attempted  to  compensate  for 
maintenance  skill  deficiencies  as  noted  above.  Compensating  for,  rather  than 
tackling  the  root-causes  of  the  problem,  however,  may  be  leading  the  Navy  down 
the  path  of  ever-decreasing  self-sufficiency  --  one  of  the  key  characteristics 
in  which  the  U.S.  Navy  has  excelled,  and  still  excels,  over  the  Soviet  Navy. 

Essentially,  what  is  happening  is  that  the  most  complex  maintenance  tasks 
(system-level  failure  diagnostics)  are  performed  by  nonorganic  technicians, 
routine  corrective  maintenance  by  senior  petty  officers,  and  preventive 
maintenance  (which  is  limited  to  simple,  proceduralized  tasks  in  the  case  of 
Navy  electronics  equipment)  by  junior  (E-4)  personnel.  The  whole  concept  of 
apprentice  training  and  skill  enhancement  on  the  job  is  defeated  because  the 
typical  graduate  from  entry-level  training  ("A"  ♦  “C"  School)  entering  the 
Fleet  is  a  novice,  not  an  apprentice.  Often,  the  pipeline  graduate  cannot 


9CINCLANTFLT  message  to  Chief  of  Naval  Technical  Training,  Subject: 
Training  of  Technical  Ratings  for  Fleet  Maintenance;  date  time  group  071410Z, 
Oct  81. 
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First,  when  poor  maintenance  performance  results  in  excessive  equipment 
downtime  (low  A0),  one  option  is  to  carry  more  equipment  in  inventory  than 
otherwise  necessary  to  meet  mission  requirements.  For  large  end  items,  such 
as  aircraft,  this  is  not  normally  an  option,  because  the  acquisition  objective 
is  based  on  force  structure  and  attrition/training  requirements. 

Nevertheless,  using  the  F-14A  aircraft  as  an  example,  the  same  mission 
capability  achieved  today  (15  squadrons  of  12  aircraft  each  with  a 
mission-capable  rate  of  58  percent  vice  the  70  percent  goal)  could  be  achieved 
with  roughly  30  less  aircraft  if  maintenance  support  met  Navy  goals.  On  this 
basis,  one  may  impute  the  cost  of  maintenance  deficiencies  for  the  F-14A  to  be 
roughly  one  billon  dollars  in  excess  procurement  costs  and  several  hundred 
million  dollars  annual  operating  and  maintenance  (O&M)  costs,  as  suggested  by 
several  other  studies.***®'7 

Second,  Fleet  support  in  the  form  of  technical  assistance  is  available 
from  Mobile  Technical  Units  (MOTUs),  Systems  Command  field  activities,  and 
contractors.  The  available  data  show  that  the  Fleet  is  relying  increasingly 
on  this  option  to  compensate  for  organic  maintenance  shortfalls:  requests  for 
assistance  have  increased  30  percent  from  1977  through  1981,  a  period  during 
which  the  actual  number  of  surface  combatants  declined.**  Moreover,  since 
1980,  a  combat  system  maintenance  team  (approximately  22  technicians)  has  been 
embarked  on  each  Pacific  Fleet  carrier  battle  group  that  deploys.  (This  team 
assists  in  combat  system  maintenance  and  provides  on-the-job  training  to 
maintenance  personnel  aboard  the  units  of  the  battle  group.  In  addition  to 
the  50-75  civilian  personnel  (Navy  and  contractor)  normally  embarked  with  each 
battle  group,  this  team  is  aboard  each  carrier  in  support  of  the  air  wing, 
providing  maintenance  assistance  for  aircraft  and  support/test  equipment. 
Another  type  of  technical  assistance,  the  so-called  "combat  system  groom,”  has 
become  standard  practice  Navy-wide.  Scheduled  periodically  by  the  Type 
Commanders,  groom  teams  provide  a  direct  work  force  to  the  ship  or  squadron  to 
correct  equipment  malfunctions  and  ensure  the  equipment  is  in  highest 
operating  condition  ("peaked")  prior  to  deployment  or  exercises.  Originally 


5F.  Fuchs  and  K.  Inaba,  "Design  for  the  Maintainer,"  Technical  Report 
NADC-79218-60  (Warminster,  Pennsylvania:  Naval  Air  Development  Center,  March 
1981). 

6F.  Fuchs,  K.  Inaba  and  F.  Siskel,  "Performance  Improvement  for  Navy 
Training  Organizations,"  Technical  Report  NAVTRAEQUIPCEN  77-C-0166  (Orlando, 
Florida:  Naval  Training  Equipment  Center,  July  1979). 

7D.  K.  McBride  and  J.  V.  Lambert,  "A  Human  Factors  Design-for- 
Mainteners  Technology  Development  Program,"  Proceedings,  Design  for 
Maintainers  Conference  (Warminster,  Pennsylvania:  Naval  Air  Development 
Center,  March  1982). 

^Unpublished  data,  Naval  Sea  Support  Center,  Atlantic  (NAVSEACENLANT) , 
Norfolk,  VA. 
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OBSERVATIONS 

Applications  of  new  technology  in  maintenance  training  equipment  and  job 
aids  will  have  a  positive  effect  on  maintenance  skills  and  performance. 
However,  the  improvements  attainable  from  such  applications  alone  appear 
limited.  To  fully  exploit  the  promise  of  new  technology  for  training  and 
aiding,  the  Navy  must  address  a  number  of  other,  fundamental  factors 
determining  maintenance  skill  acquisition  and  performance.  This  judgment  is 
based  on  the  following  observations. 

POOR  MAINTENANCE  PERFORMANCE  IS  A  SYSTEMIC  PROBLEM.  Numerous  factors,  other 
than  training  equipment  and  aids,  influence  maintenance  skill  acquisition  and 
performance.  Poor  maintenance  performance  is  a  systemic  problem  which  defies 
solution  through  piecemeal  action.  For  the  same  reason,  the  problem  is  hard 
to  solve.  Concerns  about  a  growing  gap  between  maintenance  skills  required 
and  those  available  in  the  Navy  enlisted  force  were  first  expressed  over  20 
years  ago  in  a  study  reviewing  Department  of  the  Navy  management: 

The  Department  is  encountering  extreme  difficulties  even 
at  present  to  effectively  man  the  ships  and  aircraft  of 
the  Operating  Forces  with  qualified  and  experienced 
officers  and  technically  qualified  enlisted  personnel. 

The  increasing  complexity  indicated  in  the  ships  and 
aircraft  of  the  future  Navy  will  demand  larger  numbers  of 
these  talented  and  highly  trained  people.  According  to 
the  predictions,  the  shortage  will  continue  to  increase. 

If  this  condition  persists,  the  Navy  can  well  become 
ineffectual.  The  possibilities  indicated  in  this  study 
present  a  problem  of  very  serious  proportions.4 

That  study  group  advocated  adoption  of  a  mix  of  solution  approaches, 
including:  providing  better  incentives  to  improve  retention,  improving 
personnel  programs  and  training,  and  reducing  nonessential  complexity  in  new 
weapon  systems.  These  recommendations  were  accepted  by  the  Secretary  of  the 
Navy,  but  the  effects  of  any  actions  taken  to  date  have  been  minimal;  the 
above  quotation  applies  without  change  to  today's  situation. 

SOLVING  THE  PROBLEM  IS  FAR  CHEAPER  THAN  COMPENSATING  FOR  IT.  Compensating  for 
the  effects  of  maintenance  performance  shortfalls  is  easier  than  solving  the 
problem,  and  this  is  what  the  Navy  has  traditionally  attempted  to  do. 
Compensating  actions  fall  in  three  categories:  (1)  redundancy  of  equipment, 
(2)  reliance  on  nonorganic  Fleet  support,  and  (3)  excess  supply  of  spares. 

All  of  these  are  very  expensive,  but  the  costs  are  never  attributed  to 
maintenance  deficiencies. 


*RADM  H.  J.  Kossler,  USN,  Study  Director,  "Review  of  Management  of  the 
Department  of  The  Navy,"  Volume  II,  Study  5  of  7  Studies:  Manpower  Management 
Study.  (26  October  1962)  p.  112. 
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Installations,  and  Logistics.3  Also  examined  was  the  impact  of  the  Navy's 
planned  growth  in  terms  of  electronics  maintenance  manpower  requirements  and 
the  Navy's  ability  to  meet  these  requirements  under  present  manpower 
recruitment  policies  and  known  demographic  trends.  Section  III  summarizes 
these  findings  and  conclusions  about  future  trends. 

SUMMARY  OF  RESULTS 

The  need  for  additional  R&D  in  maintenance  training  and  aiding  was 
confirmed  by  the  conclusions  drawn  as  a  result  of  this  study: 

a.  Combat  system  maintenance  performance  is  weak  and  inhibits  the  Navy 
from  achieving  peacetime  material  readiness  goals. 

b.  Lack  of  individual  maintenance  skills  is  one  of  the  top  three 
factors  causing  maintenance  performance  deficiencies;  in  turn,  inadequate 
technical  training  is  the  top-rated  factor  explaining  lack  of  skills. 

c.  Future  technology,  by  itself,  is  not  going  to  alleviate  the 
maintenance  skill  gap.  The  most  probable  trend  is  a  continued  increase  in  the 
cognitive  "load"  of  the  organizational-maintenance  job,  requiring  system-level 
understanding,  a  mental  model  of  the  symptom-failure  process,  and  cognitive 
abilities  related  to  information  processing  and  decision-making  skills. 
However,  the  intermediate-maintenance  level,  as  known  today  for  avionics, 
will,  in  all  probability,  disappear,  thereby  reducing  the  need  for 
intermediate-maintenance  technicians. 

d.  Present  manpower  recruitment  and  personnel  policies  do  not 
accommodate  the  planned  growth  of  the  Active  Navy.  Changes  will  be  required 
in  personnel  policies  to  improve  retention,  and  in  manpower  recruitment 
policies  to  maintain  the  minimum,  Congressionally  mandated,  accession  quality 
standards.  Electronics  maintenance  personnel  requirements  are  growing  at  a 
rate  more  than  double  that  Navy-wide.  Cost-effectiveness  of  technical 
training  is  becoming  the  single  most  critical  issue  in  meeting  future 
operational  requirements. 

Based  on  an  examination  of  the  factors  influencing  acceptance, 
utilization,  and  effectiveness  of  maintenance  job  aids  and  maintenance 
training  equipment,  certain  voids  in  the  current  state  of  knowledge  were 
identified.  Candidate  R&D  projects  in  maintenance  training  and  aiding 
technology  are  identified  in  Appendix  A. 


3"Steps  Toward  Improving  the  Materiel  Readiness  Posture  of  the 
Department  of  Defense,"  a  joint  Office  of  the  Secretary  of  Defense 
(OSD)-Services-Industry  study,  commissioned  by  the  Under  Secretary  of  Defense 
for  Research  and  Engineering,  managed  by  the  Institute  for  Defense  Analyses 
(IDA),  and  completed  in  September  1983.  This  30-volume  study  is  commonly 
referred  to  as  the  OSD-IDA  R&M  Study. 
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which  drive  maintenance  training/aiding  requirements  and  to  provide  an 
assessment  of  how  these  requirements  may  change  in  the  future. 

STUDY  OBJECTIVE 

The  objective  of  this  study  was  threefold: 

a.  To  define  the  factors  affecting  maintenance  requirements  and 
performance  in  the  Navy,  and  evaluate  their  impacts  and  interrelationships. 

b.  To  project  how  these  factors  may  change  in  the  future,  both 
short-term  (1980s)  and  mid-term  (1990s),  and  assess  the  impact  of  those 
changes  on  maintenance  requirements  and  performance. 

c.  To  identify  those  hey  factors  which  the  Navy's  maintenance  training 
technology  research  program  should  address  to  maximize  payoff  from  R&D 
investments. 

STUDY  APPROACH 

The  study  effort  focused  on  combat  systems  maintenance  (i.e.,  primarily 
electronics)  and  examined  three  key  questions: 

a.  Are  maintenance  performance  deficiencies  causing  a  serious  shortfall 
in  peacetime  material  readiness  or  projected  wartime  sustainability  of  the 
Navy's  mission-essential  combat  systems? 

b.  To  what  extent  is  inadequate  maintenance  skill  training  (in  formal 
schools  or  on  the  job)  the  root  cause  of  maintenance  performance  deficiencies? 

c.  Will  technology  applied  in  future  weapon  systems  eliminate  the 
maintenance  skill-deficiency  problem? 

To  examine  the  first  two  questions,  several  case  studies  were  conducted 
on  a  small  sample  of  combat  systems  currently  installed  on  air,  surface,  and 
subsurface  platforms.  Section  II  documents  the  findings  and  conclusions  on 
the  extent  of  maintenance  performance  shortfalls,  the  untoward  impact  on 
operational  availability  (A0),  and  the  "band-aid"  programs  developed  by  the 
Fleet  to  work  around  these  deficiencies.  In  order  to  aid  an  assessment  of 
cause-effect  relationships  in  maintenance  performance,  a  descriptive  model  of 
maintenance  performance  was  developed,  showing  the  interactions  of  known 
factors  directly  or  indirectly  influencing  maintenance  skill  requirements  or 
maintenance  performance,  and  their  relationships  to  A0.  Section  II 
summarizes  this  assessment  of  cause-effect  relationships  based  on  observations 
from  the  case  studies  and  a  simplified  version  of  the  descriptive  model. 

To  examine  the  third  question,  trends  in  electronics  technology  were 
reviewed  as  described  in  the  literature  and  in  a  parallel  study  sponsored 
jointly  by  the  Office  of  the  Under  Secretary  of  Defense  for  Research  and 
Engineering  and  the  Office  of  the  Assistant  Secretary  of  Defense  for  Manpower, 
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SECTION  I 
SYNOPSIS 


BACKGROUND 

The  Navy,  like  the  other  Military  Departments,  is  facing  the  fact  that 
the  human  element  has  become  the  most  critical,  problematic,  and  costly 
component  of  its  war-fighting  capability.  At  the  same  time,  the  hardware 
being  developed  and  fielded  is  becoming  increasingly  complex.  This  trend  has 
raised  concerns  about  the  Navy's  ability  to  recruit,  train,  and  retain  the 
numbers  and  quality  of  personnel  required  to  operate  and  support  the  Fleet. 
For  example,  the  former  Chief  of  Naval  Material,  Admiral  A.  J.  Whittle,  Jr., 
expressed  his  concern  as  follows: 

The  reason  that  I,  as  the  Chief  of  Naval  Material,  worry 
about  the  supply  of  people  is  that  I  worry  about  whether 
or  not  we  are  building  the  right  Navy;  whether  or  not  we 
will  be  able  to  man  the  fleet  we  are  building.1 

The  Naval  Research  Advisory  Committee  (NRAC),  commissioned  to  examine 
this  problem,  concluded  that  this  concern  was  justified: 

Given  present  trends,  the  Navy  will  find  itself  unable  to 
operate  and  maintain  its  systems,  in  either  the  short  or 
long  term,  with  the  numbers  of  skilled  personnel  necessary 
for  effective  mission  accomplishment.2 

The  NRAC  Study  Group  developed  a  number  of  recommendations  to  improve  the 
application  of  man-machine  technology  in  the  design  of  new  systems  and  the 
retrofit  of  current  systems,  to  reduce  operator  and  maintainer  task 
complexity,  to  increase  standardization,  to  improve  shipboard  productivity 
through  labor-saving  methods  and  automation,  to  enhance  shipboard  individual 
and  tactical  team  training,  and  to  increase  the  Navy's  emphasis  on  training 
systems  research  and  development  (R&D). 

As  one  response  to  the  latter  recommendation,  the  Naval  Training 
Equipment  Center  (NAVTRAEQUIPCEN)  and  the  Navy  Personnel  Research  and 
Development  Center  (NAVPERSRANDCEN)  jointly  formulated  a  Plan  of  Action  and 
Milestones  for  research  on  maintenance  training  and  aiding  (April  1981).  The 
present  study  is  one  of  the  research  efforts  identified  in  that  plan.  The 
study  is  intended  to  provide  a  better  understanding  of  the  numerous  factors 


^Office  of  the  Assistant  Secretary  of  the  Navy  (Research,  Engineering 
and  Systems),  "Man-Machine  Technology  in  the  Navy,"  NRAC  80-9,  (December  1980). 


2Ibid. 


f'lHlV'-  '  ’  V  'IW' .■V  1 


T. '  >.■  T.'  ■*." 


NAVTRAEQUIPCEN  MDA903-81-C-0166-1 


are  different  and  sometimes  inconsistent  or  incompatible  with  those  former 
criteria. 

The  case  examples  illustrate  these  points.  For  Naval  air,  the  MC  goal 
for  deployed  squadrons,  as  stated  by  the  Chief  of  Naval  Operations  (CNO),  is 
70X  for  both  the  S-3A  and  the  F-14A.  Thus,  as  long  as  the  aircraft  can  be 
flown  7 OX  of  the  time,  the  squadron  is  meeting  its  goal.  The  reported  MC 
rates  (which  vary  over  time)  are  typically  close  to  this  goal.  However, 
reported  FMC  rates  for  deployed  squadrons  are  somewhat  less  than  CNO's  goal 
which  is  defined  as  80X  of  NC  rates.  The  gap  between  HC  and  FMC  rates  (i.e., 
the  percent  of  time  aircraft  are  in  PMC  status)  is  explained  primarily  by  the 
"down"  status  of  mission  avionics.  Yet,  the  raison  d'etre  of  the  F-14A  is  its 
high-performance  radar  system  (the  AN/AWG-9);  that  of  the  S-3A,  its  avionics 
required  for  the  antisubmarine  warfare  mission.  Either  can  be  down 
permanently  without  affecting  reported  MC  rates.  Both  systems  were  acquired 
based  on  demonstrated  mission  reliability  and  re-fly  reliability  criteria. 
Applying  those  same  criteria  in  periodic  operational  readiness  evaluations 
might  provide  a  better  picture  of  the  extent  of  material  readiness  shortfalls 
than  that  provided  by  reported  MC  rates. 

For  shipboard  combat  systems/subsystems,  the  situation  is  somewhat 
different  because  A0  goals  are  specified  and  A0  achieved  is  monitored 
(insofar  as  possible  with  available  3-H  data).  But,  again,  validity  of  these 
data  is  confounded  by  the  different  operational  modes  possessed  by  these 
systems.  For  example,  the  Mk  86  Gun  Fire  Control  System  (GFCS)  has  four 
operational  modes;  in  two  modes  (direct  and  indirect  Naval  gunfire  support), 

A0  is  reaching  the  stated  goal;  in  one  (surface  mode),  A0  is  below  the 
goal  but  above  the  threshold;  and  in  one  {air  mode),  A0  is  some  50  percent 
below  the  goal.12  Visibility  of  this  shortfall  is  diminished  because,  in 
1980,  the  antiair  warfare  mission  for  many  platforms  equipped  with  the  Mk  86 
was  reduced  from  a  primary  to  secondary  mission;  i.e.,  those  surface 
combatants  can  report  "Cl"  status  for  material,  while  the  air  mode  of  the  Mk 
86  is  inoperable.  The  AN/WSC-3  Satellite  Communications  Set,  in  the 
satellite-communications  mode,  reaches  its  readiness  goal  because  only  50 
percent  of  the  multiple  units  installed  aboard  a  surface  platform  need  to  be 
operable  to  satisfy  peacetime  communications  requirements.  The  line-of-sight 
version,  however,  may  be  down.  The  AN/BQQ-5  sonar  system  has  many  different 
operational  modes  divided  among  two  major  mission  modes:  active  and  passive. 
Most  of  the  equipment  associated  with  the  active  mode  (roughly  50  percent  of 
the  system)  can  be  down  without  affecting  the  passive  mode.  Because  the 
latter  is  the  mode  commonly  used,  and  the  system  is  designed  for  "graceful 
degradation,"  the  reported  MC  rate  is  relatively  high,  especially  considering 
the  complexity  of  the  system.  Data  are  not  even  collected  on  the  up  or  down 
status  of  the  active  mode.  The  Mk  117  Fire  Control  System  (FCS)  also  has  a 
relatively  high  reported  availability  rate,  but  these  data  may  be  equally 


12This  information  is  based  on  LMI's  analysis  of  the  data  available  as 
of  December  1981. 
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misleading  because,  as  a  fire  control  system,  the  system  is  seldom  exercised 
in  peacetime. 

In  sum,  available  readiness  indicators  ("C"-ratings  and  reported  HC 
rates)  provide  neither  insight  nor  precise  measures  of  A0  shortfalls.  The 
interpretation  made,  based  on  the  analysis,  of  the  evidence  collected,  is  that 
there  are  A0  shortfalls;  that  these  shortfalls  are  serious;  but  that  the 
extent  of  the  shortfalls  is  not  adequately  reflected  in  the  readiness 
indicators.  Therefore,  the  term  A0  shortfalls  is  used  to  avoid  potential 
arguments  about  whether  or  not  reported  MC  rates  are  in  consonance  with  CNO 
goals.  Such  arguments  are  not  germaine  to  this  study. 

TRENDS  IN  OPERATIONAL  AVAILABILITY.  The  A0  demonstrated  by  the  case 
examples  exhibit  two  trends  of  interest  to  the  study.  One  trend  is  the 
increase  of  A0  with  time  since  first  Fleet  introduction.  The  current  A0 
achieved  by  each  case  example  is  much  superior  to  that  achieved  earlier. 

Three  factors  are  responsible  for  this  improvement:  improved  supply  support, 
engineering  changes  improving  the  reliability  and  maintainability 
characteristics  of  the  equipment,  and  actions  taken  by  the  Fleet  (primarily, 
technical  assistance  for  equipment  "grooming"  and  on-the-job  training  (OJT)). 
The  second  trend,  across  all  case  examples,  is  an  increase  in  A0  with 
decreasing  complexity,  with  some  notable  exceptions. 

The  term  "complexity"  refers  to  two  dimensions.  One  is  the  inherent 
complexity  of  a  system,  which  can  be  measured  in  terms  of  parts  counts  and 
interconnectivity.  Inherent  complexity  is  influenced  by  operational 
capabilities  (number  of  different  modes),  technology,  and  level  of 
integration.  The  second  dimension  is  maintenance  complexity,  the  complement 
of  testability;  i.e.,  the  percent  of  failure  symptoms  which  the  technician 
must  solve  (without  the  use  of  automated,  built-ic  diagnostics),  and  the 
difficulty  of  this  task  (number  of  paths,  number  of  steps). 

Table  2  shows  our  rank  order  of  the  equipment  reviewed  as  case  examples 
in  terms  of  inherent  complexity,  maintenance  complexity,  and  A0.  Our  rank 
ordering  in  terms  of  A0  represents  a  subjective  judgment,  however,  our 
assessment  is  based  on  inputs  received  from  the  Fleet  and  an  examination  of 
the  available  3-M  data  during  1983.  The  simplest  equipment,  the  AN/WSC-3 
satellite  communications  (SATCOM)  set,  has  the  lowest  maintenance  complexity 
and  the  highest  A0,  as  one  would  expect.  The  AN/BQQ-S  sonar  system,  on  the 
other  hand,  has  the  highest  inherent  complexity  but  scores  lower  in 
maintenance  complexity  than  most  of  the  other  examples;  as  a  result,  its  A0 
is  the  next  highest,  after  that  of  the  AN/WSC-3.  At  the  risk  of  being  accused 
of  comparing  apples  and  oranges,  the  table  illustrates  two  points: 

a.  Proper  design  for  testability  (functional  partitioning,  elimination 
of  feedback  loops,  both  in  hardware  and  software,  proper  test  point  locations, 
highly  reliable  built-in  test  (BIT)  hardware,  and  many  other  design  features) 
can  transform  an  inherently  very  complex  system  into  a  system  with  much- 
reduced  maintenance  complexity  (compare  rank  orders  in  first  two  columns). 
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TABLE  2.  OPERATIONAL  AVAILABILITY  VERSUS  COMPLEXITY 


RANK  ORDER  IN  TERMS  OF  DECREASING: 


EQUIPMENT 

j  Inherent1 

j  Complexity 

1 

Maintenance2 

Complexity 

1 

|  Operational 
j  Availability 
1 

AN/BQQ-5  Sonar 

1 

I  (D 

(5) 

1 

)  (2) 

Mk  117  FCS 

|  (2) 

(2) 

|  (3) 

Mk  86  GFCS 

|  (3) 

(1) 

|  (4) 

ASW  Avionics  (S-3A) 

}  (4) 

(3) 

|  (6) 

AN/AWG-9  Radar  (F-14A) 

1  (5) 

(4) 

|  (5) 

AN/WSC-3  SATCOM 

|  (6) 

(6) 

1  (1) 

^Module  count  and  interconnectivity . 
^Complement  of  testability. 


b.  Factors  other  than  maintenance  complexity  influence  A0  (compare 
rank  orders  in  the  last  two  columns).  These  other  factors  are  explained  next. 

RANK  ORDER  OF  CONTRIBUTING  CAUSES.  We  have  ranked  inadequate  maintenance 
performance  as  the  foremost  cause  of  the  A0  shortfalls.  Without  changing 
current  supply  support  budgets  and  equipment  reliability  characteristics,  it 
appears  possible  that  requirements  could  be  met  if  maintenance  would  be 
performed  proficiently.  Some  years  ago,  poor  supply  support  may  have  been  as 
significant  a  cause  as  poor  maintenance;  this  is  no  longer  true  today.  Much 
of  the  reported  spares  shortage  is  actually  the  result  of  poor  maintenance: 

20  percent  of  avionics  weapon  replaceable  assemblies  (WRAs)  and  60  percent  of 
shipboard  combat  system  lowest  replaceable  units  (LRUs),  replaced  by  "0"-level 
maintenance,  are  NFRs ;  SO  to  70  percent  of  the  shop  replaceable  assemblies 
(SRAs ) ,  replaced  by  "I"-level  maintenance  in  avionics  components,  are  NFRs; 
and  according  to  information  provided  during  the  Fleet  interviews, 
approximately  20  to  25  percent  of  repaired  WRAs,  and  10  to  IS  percent  of  SRAs 
and  LRUs,  do  not  work  when  installed  in  the  system.13  (The  latter  reflect 


13The  discussion  adheres  to  Navy  terminology.  Physically,  WRAs  may  be 
thought  of  as  "black  boxes";  SRAs  and  LRUs  as  circuit  boards,  wiring,  and 
other  electronic  modules.  The  term  "component"  for  WRAs  and  LRU  assemblies, 
and  "module"  for  SRAs  or  LRUs,  will  be  used  whenever  these  more  general  terms 
cannot  cause  confusion.  The  NFR  rates  reported  here  are  based  on  the  case 
examples.  They  differ  slightly  from  those  noted  in  Section  I,  which  are  based 
on  Navy-wide  data  as  reported  by  the  Navy. 
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inadequate  repairs  and/or  poor  quality  control  at  "D"-level  or  vendors,  which 
is  partly  due  to  limitations  of  the  shop  test  equipment;  e.g.,  inconsistent 
tolerances  and  inability  to  simulate  the  dynamic  operational  environment  by  a 
static  testing  approach.)  The  elimination  of  these  inefficiencies  could 
nearly  halve  the  requirement  for  spares. 

This  assessment  does  not  imply  that  equipment  reliability  is 
unimportant.  Quite  the  contrary:  by  virtue  of  the  definition  of  A0, 
reliability  is  the  most  important  factor  influencing  A0.  Low  reliability 
must  result  in  low  availability,  even  with  perfect  maintenance,  and  the  higher 
the  reliability,  the  less  impact  logistic  resources  have  on  A0.  The  point 
is  that  the  demonstrated  A0  shortfalls  for  Navy  combat  systems  are  due 
chiefly  to  inadequate  maintenance  performance,  much  less  so  to  lower 
reliability  than  specified,  less  supply  support  than  needed,  or  any 
shortcomings  in  Fleet  support. 

In  turn,  inadequate  maintenance  performance  is  caused  by  many  factors 
(refer  back  to  Figure  1).  The  contributions  of  these  factors  cannot  be 
quantified  exactly  from  the  available  data,  but  can  be  ranked  in  order  of 
importance.  Table  3  shows  the  rank  order  found  for  each  example  based  on  our 
analysis  of  the  available  3-M  data  and  Fleet  interviews  we  conducted  during 
1983.  The  top-rated  factor  differs  from  case  to  case;  for  some  (e.g., 
avionics),  the  operational  environment  is  the  most  important  factor 
responsible  for  poor  maintenance  performance;  for  others,  it  is  technical 
training.  Fleet  practices,  or  poor  equipment  design  (maintainability). 

TABLE  3.  RANK  ORDER  OF  FACTORS  CAUSING  POOR  MAINTENANCE  PERFORMANCE 
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Overall,  across  all  case  examples,  we  have  rank  ordered  the  factors 
contributing  to  inadequate  maintenance  performance  as  follows: 

1.  Technical  Training.  Inadequate  maintenance  skills  is  the  first 
factor.  This  deficiency  is  attributed  mostly  to  lack  of  individual 
proficiency  which,  in  turn,  is  attributed  chiefly  to  inadequate  training. 
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2.  Maintainability  Characteristics.  Poor  design  for  maintainability, 

especially  manifested  in  poor  testability,  is  the  second  most  important 
contributor  to  poor  maintenance  performance.  > - 

3.  Fleet  Practices.  Excessive  diversions  fro9;^6  maintenance  job,  and 
especially  the  practice  of  using  school  graduates  for  mess  duty  during  their 
first  three  to  six  months,  detract  from  maintenance  performance.  The  impact, 
however,  is  more  a  delay  in  shill  acquisition.  More  important  is  the  fact 
that  maintenance  is  accorded  a  much  lower  priority  than,  operations,  which,  in 
selected  cases,  inhibits  proper  maintenance. 

4.  Job  Design.  The  traditional  allocation  of  maintenance  tasks  among 
different  ratings  and  the  absence  of  system-levei  technicians  is  a  major 
factor  explaining  poor  maintenance  performance  in  the  case  of  integrated 
systems . 

5.  Personnel/Experience .  The  shortage  of  experienced  technicians 
(percent  fill  of  authorized  billets  for  petty  officers  by  rating/Navy  Enlisted 
Classification  (NEC))  is  the  fifth  factor. 

6.  Technical  Hanuals/Job  Aids.  The  next  factor  is  shortcomings  in 
technical  information.  Shortcomings  include:  inaccurate,  incomplete,  or 
out-of-date  information;  and  difficult  to  interpret,  poor  accessibility  to,  or 
the  inability  to  use  manuals  at  the  job  site. 

7.  Spares.  Spares  shortages  have  a  direct  influence  on  A0,  but  also 
affect  maintenance  performance.  Shortages  may  cause  additional  maintenance 
downtime  (e.g.,  workaround  procedures),  additional  maintenance  actions  (e.g., 
for  fault  isolation  and  verification),  cannibalization  (to  place  the  "hole" 
where  it  hurts  least;  e.g.,  by  removing  a  needed  spare  from  another  system 
which  is  already  "down"  for  other  reasons),  or  emergency  repairs  (doing 
maintenance  normally  assigned  to  higher  echelons,  which  by  lack  of  tools, 
parts,  and  training,  invariably  affects  the  integrity  of  the  system).  In 
short,  lack  of  needed  spares  makes  the  maintenance  task  more  difficult. 

8.  Tools/Test  Equipment.  Lack  of  available,  calibrated  test  equipment 
is  the  next  factor.  As  in  the  case  of  spare  parts,  part  of  the  test  equipment 
problem  is  caused  by  inefficient  maintenance  at  higher  echelons  and/or  user- 
inflicted  damage. 

9.  Operational  Environment.  Operational  tempo  (time  available  for 
maintenance  turnaround)  is  the  most  important  factor  in  the  case  of  avionics 
maintenance  aboard  deployed  carriers.  For  nondeployed  units,  this  factor  has 
little  impact  on  maintenance  performance.  For  the  mix  of  equipment  examined, 
this  factor  rates  ninth  overall  (Navy-wide). 

10.  Maintenance  Concept.  The  maintenance  concept  for  nonavionics 
electronics  equipment  is  essentially  two-level.  Effectiveness  of  this  concept 
depends  on  supply  support.  Ship's  force  is  frequently  required  to  repair 
electronic  modules  because  it  is  short  of  spares,  but  such  emergency  repairs 
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are  hindered  by  lack  of  piece  parts  and  test  equipment.  (The  Navy  is 
currently  evaluating  a  better-managed  approach  to  emergency  repairs.) 

11.  Operator  Skills.  For  most  shipboard  systems,  the  operator  tasks  and 
maintenance  tasks  are  combined  into  one  job  (billet).  One  notable  exception 
is  surface-ship  communications  equipment,  operated  by  the  Radioman  (RM) 
rating,  maintained  by  the  Electronics  Technician  (ET)  rating.  In  this  case, 
the  job  design  approach  has  resulted  in  maintenance  problems  due  to  excessive 
operator-inflicted  damage. 

Two  factors  identified  in  the  model  are  missing  from  the  above  listing: 
individual  abilities  and  equipment  reliability.  The  case  studies  did  not 
permit  a  quantitative  assessment  of  individual  abilities,  therefore  it  is  not 
known  how  important  they  are  with  regard  to  maintenance  performance. 
Reliability  has  little  impact  on  maintenance  performance.  It  determines  the 
maintenance  workload,  which  supposedly  determines  maintenance  manning 
requirements  and  authorizations.  A  potentially  important  factor  is  the 
discrepancy  between  planned  and  actual  reliability  achieved  in  the  operational 
environment.  While  the  case  examples  exhibit  substantial  discrepancies,  the 
additional  workload  is  judged  well  within  maintenance  capabilities  if 
maintenance  was  performed  proficiently.  In  short,  the  impact  of  reliability 
is  primarily  on  A0,  not  on  maintenance  performance. 

The  above  listing  applies  to  "0"-level  maintenance  performance,  the 
subject  on  which  this  study  concentrates  most.  With  respect  to  "I"-level 
maintenance,  only  that  maintenance  performed  afloat  aboard  carriers  by  the 
Aircraft  Intermediate  Maintenance  Department  (AIMD)  was  examined.  "I"-level 
maintenance  performed  by  intermediate  maintenance  activities  such  as  tenders, 
or  shore  intermediate  maintenance  activities  (SIMAs),  was  beyond  the  scope  of 
this  study  because  such  activities  are  little  involved  in  combat  system 
maintenance  or  repairs  of  combat  systems  components  or  modules.  All  shipboard 
combat  system  have  basically  a  two-level  maintenance  concept,  "0"-  and 
"D"-level;  most  avionics  equipment  has  a  three-level  maintenance  concept, 

"0"-,  "I"-,  and  "D"-level,  where  "I"-level  is  exclusively  performed  by  the 
AIMD. 


"I"-level  maintenance  is  as  troublesome  as  that  performed  at  the 
"0"-level,  but  it  has  a  less  immediate  impact  on  A0.  The  chief  contributing 
factor  is  the  lack  of  operational  suitability  of  much  of  the  ATE  currently 
installed  (i.e.,  poor  reliability,  poor  maintainability,  and  incompatibility 
with  most  of  the  avionics  components  and  modules  so  that  complex  interface 
devices  must  be  used)  and  the  limited  availability  and  poor  quality 
(diagnostic  performance)  of  the  test  program  sets.  Inadequate  training  of  ATE 
operators,  maintainers,  and  software  analysts  is  a  distant  second  factor.  All 
of  the  other  factors  are  negligible  in  comparison  to  these  two. 

ANALYSIS  OF  CONTRIBUTING  CAUSES.  Each  case  example  has  a  different  story  to 
tell.  There  is,  however,  a  common  thread  which  may  explain  why  the  Fleet  has 
so  much  trouble  meeting  realistic  readiness  requirements  in  peacetime.  While 
thi3  assessment  is  based  on  a  limited  sample,  and  the  specific  rank  order  of 
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contributing  factors  may  differ  from  case-to-case  as  well  as  over  time,  this 
common  thread  appears  to  be  generic  to  the  current  weapon  system  acquisition 
and  logistic  support  processes  and,  therefore,  may  be  more  informative  in 
identifying  root-causes  vice  symptoms  of  £h*.-  problem. 

The  common  thread  consists  of  a  series  of  assumptions  made  in  weapon 
system  acquisition  and  support  —  assumptions  which  appear  invalid  and,  in 
combination,  result  in  a  gap  between  aaintetoanpe  skill  requirements  and  the 
skills  possessed  by  Navy  technicians: 

a.  Design  Concept.  Combat  system  design  concepts  invariably  emphasize 
minimum  manning  and  skill  level  requirements.  This  is  achieved  through 
modular  design  and  BIT.  No  contingency  plans  are  made  in  case  the  concept 
turns  out  to  be  unrealistic.  The  process success-driven;  any  problems  are 
assumed  to  be  subject  to  solution  through' engineering  changes. 

b.  Contract.  The  contract  between  the  government  and  the  prime 
contractor  specifies  the  design  requirements.  Vith  respect  to 
maintainability,  if  any  quantitative  requirements  for  testability  are 
included,  the  measures  used  are  arcane;  i.e.,  they  are  ambiguous  and  open  to 
different  interpretations,  they  cannot  be  tested  or  evaluated,  and  they  lack 
meaning  in  the  operational  environment.  Nevertheless,  the  assumption  is  that 
the  contract  specifies  precisely  the  requirement. 

c.  Design  Engineering.  In  attempting  to  meet  the  numeric  testability 
requirements,  current  design  engineers  customarily  make  three  critical 
assumptions:  (1)  all  faults  are  catastrophic,  i.e.,  a  fault  is  a  fault;  (2) 
only  single  faults  occur;  and  (3)  faults  occur  only  within  the  electronic 
modules,  not  in  interconnecting  wires  or  connectors.  None  of  these 
assumptions  are  valid. 

d.  Maintenance  Engineering.  Responsibility  for  logistics  support 
analysis  (LSA)  is  a  maintenance  engineering  function,  which  may  or  may  not  be 
integrated  with  design  engineering.  In  either  case,  the  LSA  is  dependent  upon 
the  design  engineer's  specifications.  Because  the  latter  are  not  realistic, 
the  LSA  records  are  not  either;  nevertheless,  they  are  assumed  to  identify  all 
maintenance  task  requirements  at  each  level  of  maintenance. 

e.  Job-Task  Analysis.  If  an  early  analysis  is  conducted  for  developing 
training  requirements,  much  of  the  "subject  matter  expert's"  knowledge  of  the 
new  system  must  come  from  the  contract  specifications.  Unless  gifted  by  a 
great  deal  of  skepticism  (acquired  through  experience),  the  subject  matter 
expert's  analysis  will  indicate  that  most,  if  not  all,  of  the  fault 
diagnostics  are  provided  by  BIT.  The  resulting  job-task  list,  which 
eliminates  most  nonprocedural  tasks,  is  assumed  to  identify  the  technician's 
job-task  requirements.  The  same  result  is  obtained  when  the  analysis  is 
deferred  and  based  on  completed  LSA  records, 
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f.  Instructional  Systems  Development  (ISP).  The  ISO  model,  as 
implemented  by  the  Navy,  is  designed  only  for  procedure  training,  not  for 
development  of  cognitive  skills.  The  model  addresses  only  those  behavioral 
tasks  in  the  job-task  list  which  yield  to  its  methodology,  systematically 
excluding  any  information  that  is  not  directly  task-specific  or  cannot  be 
taught  in  a  stimulus-response  training  process.  In  spite  of  these  fundamental 
limitations,  the  ISD  model  is  perceived  as  an  effective  approach  for 
developing  training  courses. 

g.  Training.  The  training  plan  and  program  of  instruction  are  based  on 
teaching  that  subset  of  identified  tasks  which  can  be  most  effectively  taught 
in  formal  school  within  budgetary  constraints,  considering  the  following  four 
factors:  task-learning  difficulty,  importance  of  the  task  (frequency  of 
performance  or  time  spent  performing),  criticality  of  the  task  (consequences 
of  inadequate  performance;  i.e.,  primarily  personnel  safety),  and  immediacy  of 
task  performance  (task-delay  tolerance).  The  assumption  is  that  the  rest  will 
be  learned  (somehow)  on  the  job.  The  present  training  philosophy  does  not 
provide  for  follow-on,  career-level  training. 

h.  Maintenance  Demonstrations  and  Operational  Testing.  Maintenance 
demonstrations  are  conducted  with  predefined  system  failures.  Operational 
testing  is  of  limited  duration.  Neither  is  sufficient  for  assessing  BIT 
performance  or  the  completeness  and  validity  of  the  job-task  list.  Even  if 
BIT  shortfalls  are  identified,  approval  for  Service  use  is  invariably  granted 
(if  the  system's  operational  capability  is  satisfactory)  in  order  to  meet 
Fleet  operational  requirements.  Navy  technicians  participating  in  operational 
tests  are,  normally,  factory-trained,  so  that  shortfalls  in  Navy  technical 
training  do  not  surface. 

i.  Fleet  Experience.  Once  the  system  finally  reaches  the  Fleet, 
technicians  encounter  problems  troubleshooting  with  BIT  such  as: 

(1)  false  alarms, 

(2)  erroneous  fault-isolation  indications, 

(3)  a  higher  percentage  than  advertised  not  isolated  at  all, 

(4)  high  ambiguity  of  BIT  indications, 

(5)  excessive  time  required  in  using  operator-interactive 
diagnostics . 

Actually,  the  BIT  is  normally  very  accurate  in  what  it  is  designed 
to  do;  the  problem  is  that  many  failures  (in  excess  of  50  percent  according  to 
most  studies  reviewed)  do  not  meet  the  design  assumptions.  The  BIT,  however, 
cannot  discriminate  between  failures  it  is  designed  to  address  and  those  it  is 
not.  Four  additional  problems  serve  to  detract  further  from  operational  BIT 
performance.  One  is  that  the  level  of  testability  is  designed  to  meet  stated 
requirements  using  predicted  failure  modes  and  rates.  To  the  extent  actual 


22 


NAVTRAEQUIPCEN  HDA903-81-C-0166-1 


reliability  behavior  differs  from  the  reliability  engineering  model,  BIT 
performance  (measured  in  terms  of  percentages  of  actual  failures)  is  always 
less  than  designed.  The  second  problem  is  that  the  gap  between  advertised  and 
actual  BIT  performance  increases  over  time;  due  to  frequent  removal/ 
replacement  of  modules,  the  percentage  of  failures  occurring  in  interfaces 
grows  over  time,  and  these  are  the  failures  normally  not  accounted  for  in  BIT 
design.  The  third  problem  is  caused  by  failures  within  the  BIT  hardware  or 
software.  Specifications  are  normally  careful  in  requiring  that  BIT  failures 
should  not  affect  prime  equipment  operational  status,  but  BIT  failures  are 
often  not  discernible  as  such  to  the  technician.  The  result  is  that  the 
equipment  must  be  taken  down  for  maintenance,  even  though  BIT  failures 
formally  do  not  affect  operational  capabilities.  When  this  happens  several 
times,  the  technician  loses  confidence  in  the  BIT.  The  fourth  problem  is  that 
special  diagnostic  programs,  requiring  operator  interaction  for  fault 
isolation,  are  often  designed  without  regard  to  the  Fleet  environment,  which 
puts  a  high  premium  on  quick  repairs  of  combat-essential  equipment.  In  many 
cases,  the  testability  features  cannot  be  used  in  that  environment  because 
they  take  too  much  time,  necessitating  shortcuts  by  the  technician. 

j.  Feedback.  The  Navy's  maintenance  data-collection  system  is  not 
designed  to  identify  the  true  causes  of  maintenance  problems.  What  the  data 
show  is  that  there  is  a  problem  (large  numbers  of  CASREPs,  low  reported  FHC 
rate,  and/or  low  computed  A0)  and  that  the  Fleet  is  constantly  short  of 
spare  parts.  The  assumption  is  that  these  surface  manifestations  reveal  the 
cause  of  the  problem  as  one  of  weak  supply  support.  The  feedback  systems 
established  to  evaluate  adequacy  of  training  are  not  effective  in  identifying 
training  deficiencies. 

k.  Corrective  Action.  Based  on  the  available  information,  the  emphasis 
by  the  Systems  Commands  or  program  (project)  managers  is  understandably  on 
getting  funding  for  engineering  changes  to  improve  reliability  and  for 
increased  spares.  In  some  cases,  where  engineering  studies  (conducted  by 
contractors  as  part  of  post-production  support)  identify  failure  diagnostic 
shortfalls,  attempts  are  made  to  improve  BIT.  But,  for  practical  reasons, 
such  improvements  are  normally  limited  to  software,  deferring  more  costly 
hardware  improvements  to  a  later  model. 

The  nature  of  the  process  is  such  that  the  root  problem  is  not  attacked. 
Particularly,  equipping  the  technician  to  cope  with  BIT  limitations  by 
increasing  the  skill  levels  (paygrades)  or  revising  the  training  or  improving 
the  job  aids  is  not  controlled  by  anyone.  These  issues  are  addressed  again  in 
Section  IV  in  discussing  what  needs  to  be  done. 

FLEET  ASSESSMENT 

The  Fleet  has  become  increasingly  concerned  in  recent  years  about  the 
decreasing  level  of  basic  knowledge  and  skill  proficiency  of  journeyman  and 
apprentice-level  technicians  responsible  for  combat  systems  maintenance.  This 
concern  was  expressed,  for  example,  by  the  Commander  in  Chief,  U.S.  Atlantic 
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Fleet  (CINCLANTFLT)  to  the  technical  training  community.14  In  this  message, 
CINCLANTFLT  attributes  skill  deficiencies  of  journeyman  technicians  to  poor 
retention,  and  those  of  apprentice-level  technicians  to  shortfalls  in  formal 
school  training,  especially  "A"  Schools  (curriculum  content,  method  of 
instruction,  or  both).  The  Fleet  expects  apprentice-technicians,  not  novices, 
from  the  entry-level  training  pipeline;  i.e.,  they  must  be  capable  of 
performing  routine  maintenance  tasks  under  limited  supervision.  Yet,  the 
message  notes: 

Today's  apprentice-technicians  are  generally  incapable  of 
carrying  out  the  basic  tasks  related  to  equipment 
maintenance.  As  a  result,  the  senior  petty  officer  is 
required  to  fulfill  these  basic  tasks  and  outside 
assistance  must  be  called  in  to  accomplish  journeyman 
petty  officer  tasks. 

In  order  to  make  a  general  assessment  of  the  Fleet's  perception  of  the 
shortfalls  of  pipeline  graduates,  several  informal  interviews  were  conducted 
during  the  course  of  this  study.  During  these  interviews,  the  Fleet 
identified  weaknesses  of  "A"  School  graduates  in  the  following  areas: 

a.  safety  procedures, 

b.  basic  electricity  and  electronics  theory, 

c.  use  of  basic  tools, 

d.  proper  use  of  test  equipment, 

e.  troubleshooting  techniques, 

f.  printed  circuit  (PC)  board  repair, 

g.  knowledge  of  ancillary  equipment, 

h.  proper  use  of  technical  documentation, 

i.  supply  procedures. 

These  shortfalls  indicate  that  the  instructional  objectives  established 
for  the  various  schools  do  not  match  the  skills  and  knowledge  required  by  the 
Fleet.  In  turn,  this  suggests  inadequate  job-task  analysis,  inadequate 
specification  of  training  needs,  inadequate  translation  of  training  needs  into 
instructional  objectives,  inadequate  training  design,  inadequate  training 


1aCINCLANTFLT  message  to  Chief  of  Naval  Technical  Training,  Subject: 
Training  of  Technical  Ratings  for  Fleet  Maintenance;  date  time  group  071410Z 
OCT  81. 
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implementation,  or  all  of  the  above.  Importantly,  once  mistakes  have  been 
made  in  the  definition  of  training  needs  (as  a  result  of  improper  job-task  and 
skill  analysis),  any  subsequent  evaluation  of  the  training  program  is,  by 
definition,  faulty  by  lack  of  the  proper  criterion. 

IMPACT  ON  THE  FLEET.  The  impact  of  these  training  shortfalls  on  the  Fleet  is 
severe.  Short  term  ,  as  highlighted  in  the  above  CINCLANTFLT  message,  the 
direct  impact  can  be  measured  by  such  indicators  as  follows: 

a.  Increased  number  of  outstanding  CASREPs  in  combat  systems  equipment. 

b.  Significant  increase  in  requests  for  technical  assistance  (HOTUs, 
Naval  Sea  Systems  Command  (NAVSEA)  and  Naval  Electronic  Systems  Command 
(NAVELEX)  field  activities,  and  vendors). 

c.  Significant  increase  in  requests  for  system  "grooms." 

d.  Dramatic  increase  in  damaged  or  abused  general  purpose  electronic 
test  equipment  received  at  calibration  facilities. 

e.  Increased  number  of  functional  electronic  components  incorrectly 
classified  by  technicians  as  being  faulty.  For  example,  70  percent  of  the  PC 
boards  turned  into  the  Support  and  Test  Equipment  Engineering  Program  for 
repair  are  determined  to  be  functioning  correctly. 

The  increase  in  technical  assistance  required  from  the  Fleet  support  community 
is  estimated  at  30  percent  over  the  1977  to  1981  time  period,  a  period  during 
which  the  actual  number  of  surface  combatants  declined.15 

Long  term,  the  impact  is  even  more  serious:  with  senior  petty  officers 
doing  most  of  the  routine  maintenance  tasks  and  outsiders  (technical 
assistance  teams  or  embarked  civilian  technicians)  doing  the  more  complex 
corrective  maintenance  tasks,  the  whole  concept  of  apprentice  training  through 
OJT  suffers.  Apprentice-technicians  do  not  receive  sufficient,  guided, 
hands-on  practice  to  develop  maintenance  skills;  similarly,  the  skills  of 
journeyman  technicians  do  not  increase  but  stagnate.  The  long-term  impact,  in 
other  words,  it  one  of  declining  organic  skill  proficiency  in  the  Fleet  --  a 
decline  across  the  board  in  a  period  during  which  combat  systems  are  known  to 
become  more  complex  and  difficult  to  troubleshoot.  As  a  consequence,  further 
increases  in  the  extent  to  which  the  Fleet  must  rely  on  nonorganic  support 
will  be  unavoidable. 

The  available  data  suggest  that  the  above  indicators  of  poor  maintenance 
performance  are  due  more  to  formal  school  training  shortfalls  than  other 
potential  root-causes  (e.g.,  operating  tempo,  low  retention  of  journeyman 


15Unpublished  data,  Naval  Sea  Support  Center,  Altantic  (NAVSEACENLANT) , 
Norfolk,  VA. 
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technicians,  operational  environment).  For  example,  based  on  the  interviews 
with  the  Fleet,  the  following  represents  the  consensus  regarding  the  knowledge 
and  skills  possessed  by  graduates  from  the  entry-level  training  pipeline  (in 
the  ratings  responsible  for  electronics  maintenance): 

a.  Only  five  to  ten  percent  are  able  to  use  common  basic  test  equipment. 

b.  Basic  electronic  theory  is  not  understood  by  95  percent. 

c.  Host  are  able  to  run  the  built-in  diagnostic  checks  associated  with 
combat  system  preventive  maintenance,  but  85  percent  have  not  the  slightest 
idea  what  to  do  when  a  failure  is  detected  and  the  automated  diagnostics  fail 
to  isolate  the  failure  to  a  single  replaceable  module. 

Journeyman  technicians  (PO  II  or  E-5)  who  perform  the  bulk  of  corrective 
maintenance  also  exhibit  limited  troubleshooting  skills  as  evidenced  by  the 
high  retest  okay  rate  of  modules  replaced  in  the  combat  systems  and  evacuated 
to  "I"-  or  "D"-level  maintenance  activities:  about  30  to  50  percent  of  the 
modules  —  60  to  80  percent  of  PC  boards  (depending  on  combat  system)  —  are 
found  ready  for  issue;  i.e.,  were  unnecessarily  removed  from  the  shipboard 
system.16  These  percentages  are  high  considering  that  a  significant 
proportion  of  the  complex  troubleshooting  jobs  is  actually  performed  by 
so-called  experts  (technical  assistance  teams  or  embarked  civilian 
technicians).  Of  course,  these  data  must  be  interpreted  with  care:  factors 
other  than  training  may  be  the  root-cause.  The  case  studies  indicate  that 
equipment  design  characteristics  and  operational  environment  are  as  much  to 
blame  as  inadequate  training. 

The  Fleet,  of  course,  cannot  operate  under  these  circumstances.  The 
steps  which  have  been  taken  to  overcome  or  reduce  organic  skill  deficiencies 
cover  three  broad  areas:  (1)  establishment  of  an  intensive  "waterfront" 
training  activity,  (2)  an  institutionalized  program  of  combat  system  grooms 
prior  to  deployment,  and  (3)  embarkation  of  "super  technicians"  (military, 

Navy  civilians,  and  contractor  technicians)  aboard  deploying  units. 

The  Fleet  Training  Commands  and  MOTUs  have  initiated  functional  skill 
courses  in  the  deficient  areas  listed  earlier.  The  original  mission  of  MOTlIs 
was,  of  course,  to  provide  OJT  in  electronics  maintenance.  Now  classes  are 
run  around  the  clock  at  the  expense  of  the  OJT  mission. 

Combat  system  grooms  are  accomplished  by  various  equipment-oriented 
electronic  groom  teams,  consisting  of  experienced  military  and  contractor 
maintenance  technicians.  These  teams  perform  maintenance  tasks  originally 
intended  to  be  accomplished  by  organic  (ship's  force  or  air  squadron) 


1 6CINCLANTFLT  Message  to  Chief  of  Naval  Technical  Training,  Subject: 
Training  of  Technical  Ratings  for  Fleet  Maintenence;  date  time  group  0714102, 
Oct  81. 
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maintenance  personnel.  Scheduled  periodically  by  the  Type  Commander,  the 
groom  teams  provide  a  direct  workforce  to  the  ship  or  squadron  for  correction 
of  particular  equipment  deficiencies.  The  groom  team's  mission  is  to  have  the 
equipment  in  the  highest  operational  readiness  state  possible.  These  programs 
are  costly  and  manpower  intensive.  Originally  designed  as  emergency  programs, 
they  have  become  permanent  fixtures.  The  payoffs  from  these  combat  system 
grooms  are  considerable.  For  example,  F-14  missile-firing  exercises  in  the 
Caribbean  were,  in  the  past,  plagued  by  problems  first  attributed  to  operator 
skill  deficiencies.  Once  the  combat  system  groom  was  instituted,  it  was  found 
that  the  true  problem  was  one  of  poorly  aligned  equipment:  the  missile  hit 
rate  currently  achieved  is  triple  that  achieved  before  the  groom  was  a 
formally  scheduled  requirement.*7 

The  third  area,  embarkation  of  technicians  supplementing  organic  manning 
of  a  battle  group,  is  especially  pronounced  in  the  Pacific  Fleet,  not  in  the 
Atlantic  Fleet.  The  explanation  is  the  different  operational  environment: 
the  former  is  deployed  across  large  distances  with  little  forward  deployment 
of  logistic  support  resources  while  the  latter  has  considerable  forward 
support  available.  Battle  groups  deployed  with  the  Pacific  Fleet  have  about 
22  combat  system  technicians  embarked  to  provide  the  technical  skills  which 
are  otherwise  lacking.  This  number  is  in  addition  to  the  50  to  75  civilian 
technicians  embarked  aboard  each  carrier  in  support  of  the  air  wing. 

Importantly,  t  ie  success  of  these  programs  obscures  .'.he  failure  of  the 
training  pipeline  to  provide  the  Fleet  with  the  apprentice-level  technicians 
needed.  A  brief  description  of  each  Fleet‘s  corrective-action  programs 
f  ollows . 

ATLANTIC  FLEET  PROGRAM.  The  Type  Commander, - COMNAVSURFLANT,  noted  in  recent 
years  that  the  training  readiness  of  combat  system  personnel  was  seriously 
lagging  equipment  readiness,  based  on  the  following  observations: 

a.  Limited  ship's  force  involvement  in  combat  system  refurbishment  and 
repair  throughout  the  regular  overhaul  of  a  ship. 

b.  Combat  systems  not  fully  tested  or  operational  aboard  ships 
completing  regular  overhaul  or  scheduled  restricted  availabilities. 

c.  Ship's  force  inadequately  trained  to  operate,  test,  or  maintain 
combat  systems. 

d.  Unsatisfactory  operational  readiness  training  due  to  inadequate 
basic  operator  skills,  degraded  comb&c  systems,  and  equipment  casualties. 

e.  Degraded  operational  readiness  of  deployed  units  due  to  combat 
system  casualties  involving  excessive  downtimes  for  maintenance. 


17COMNAVAIRLANT  Interview,  Norfolk,  VA,  October  1982. 
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The  problem  was  attributed  to  two  root-causes.  First,  the  ship's  force 
has  a  heavy  workload  during  overhaul  so  that  training  receives  a  low  priority 
and  technical  training  is  neglected  altogether.  Second,  there  iB  no  single 
source  document  which  defines  the  minimum  standards  of  technical  training 
required  for  combat  systems  technicians:  equipment  technical  manuals, 
personnel  qualification  standards,  personnel  advancement  requirements,  and 
planned  maintenance  system  maintenance  requirements  cards  all  address 
different  aspects  of  the  technicians'  job  and  skill/knowledge  requirements. 

In  spite  of  this  mountain  of  data,  the  Navy  simply  does  not  have  a  single, 
comprehensive  document  that  provides  a  complete  and  realistic  specification  of 
the  job-task  requirements  --  a  lack  which  defeats  efforts  to  conduct  an 
effective  job  training  program. 

That  was  COMNAVSURFLANT ' s  diagnosis  of  the  true  problem.  Consequently, 
the  Combat  Systems  Improvement  Program  (CSIP)  was  instituted  in  April  1982  to 
correct  the  problem.  While  CSIP  covers  a  broad  range  of  actions,  the  focus 
here  is  on  the  training  aspects,  the  highest  payoff  area  (according  to 
COMNAVSURFLANT)  in  terms  of  expected  improvements  in  combat  systems 
operational  readiness.  In  the  training  area,  CSIP  consists  of  two  major 
initiatives:  development  of  Combat  System  Training  Requirements  Manuals  to 
identify  training  requirements,  and  establishment  of  Combat  System  Mobile 
Training  Teams  to  provide  supervised  OJT  programs  to  technicians  aboard  ships 
in  the  home  ports.  The  manuals  are  being  developed  by  specific  ship  class  and 
are  intended  to  identify  minimum  knowledge  and  skills  required  by  rating  and 
rate,  including  references  to  pertinent  source  documents.  The  plan  is  to 
evaluate  these  manuals  during  fiscal  year  1983  (FY83).  The  Mobile  Training 
Teams  consist  of  senior  combat  system  technicians  from  Group  and  Squadron 
staff  located  in  a  particular  home  port.  These  subject  matter  experts  will 
conduct  formalized  training  once  the  manuals  are  completed. 

Each  individual  ship  has  a  Ship  Electronics  Readiness  Team  (SERT)  which 
is  comprised  of  the  assigned  senior  technicians  responsible  for  each  of  the 
combat  system  components.  The  SERT  was  originally  designed  as  a  brain¬ 
storming  group  to  help  minimize  equipment  downtimes  by  cross-fertilization  of 
maintenance  and  troubleshooting  techniques.  Although  a  good  idea,  the  concept 
did  not  work  on  small  combatants  (destroyers  and  frigates)  because  the  s'  ior 
technicians  on  those  platforms  do  not  necessarily  have  much  maintenance 
experience.  Due  to  the  way  formal  school  training  is  currently  structured  (no 
formal  school  career  training  after  first  reenlistment),  there  is  little  or  no 
opportunity  for  those  technicians  to  acquire  higher  maintenance  skills.  The 
Mobile  Training  Team  concept  attempts  to  fill  this  void.  In  particular,  the 
Combat  System  Mobile  Training  Team  will  train  the  SERT  to  conduct,  monitor, 
and  evaluate  combat  system  component  testing  conducted  by  shipyard  and  vendor 
personnel,  and  to  perform  the  Overall  Combat  Systems  Operability  Test.  This 
test  exercises  all  shipboard  combat  systems'  components  and  interfaces,  and 
provides  a  measure  of  overall  combat  system  availability  and  operability  in 
accordance  with  design  specifications.  The  Mobile  Training  Team  will  also 
train  apprentice-level  technicians  in  the  area  of  diagnostic  readouts  and  in 
troubleshooting  techniques  which  are  required  to  correct  any  deficiencies 
f  ound . 
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Quantitative  Research  on  Retention  Factors  has  been  Limited.  Past  research 
has  focused  primarily  on  recruiting  issues.  Only  very  recently  has  DoD 
recognized  that  retention  may  be  more  of  a  problem  than  recruiting,  and  that 
policies  adopted  to  alleviate  recruiting  shortfalls  may  have  an  adverse  'rapact 
on  retention  (especially,  the  Variable  Education  Assistance  Plan  that  replaced 
the  GI-Bill  in  1977).  As  a  result,  little  quantitative  data  are  available  on 
factors  that  could  be  manipulated  to  increase  retention.  The  Navy  has, 
traditionally,  assumed  a  leadership  role,  compared  to  the  other  Services,  in 
examining  the  factors  (attitudinal ,  biographic,  and  economic)  influencing 
reenlistment  decisions  and  the  interactions  among  these  factors.  The  Navy  has 
introduced  two  well-designed  programs  (Human  Resource  Management  Program;  and 
Leadership,  Management  Education,  and  Training  Program)  to  enhance  motivation 
and  retention.  These  programs  receive  management  emphasis  and  attention.  In 
addition,  the  Navy  is  making  extensive  use  of  the  selective  reenlistment  bonus 
program  for  critical  skills.  Navy  spending  on  bonus  payments  is  already  about 
50  percent  of  the  amount  spent  DoD-wide.  In  sum,  the  Navy  is  doing  about  all 
that  can  be  reasonably  expected  in  the  area  of  retention. 

Nonetheless,  petty  officer  shortages  have  been  a  constant  phenomenon  in 
the  Navy.  The  Navy's  steady-state  personnel  retention  requirement  is 
approximately  as  follows:  45  percent  first-term  reenlistment,  60  percent 
second-term  reenlistment,  95  percent  third-term  reenlistment  and  beyond.45 
These  rates  had  never  been  achieved  prior  to  FY82.  In  the  late  1970s,  the 
rates  achieved  were  37  to  40  percent  for  first-term  reenlistments,  45  to  53 
percent  for  second-term  reenlistments,  and  91  to  93  percent  for  subsequent 
reenlistments.46  The  result  was  a  shortage  in  the  petty  officer  ranks  that 
will  persist  through  the  1980s  in  spite  of  the  improved  retention  results  in 
FY82  and  FY83.  For  example,  the  Navy's  manpower  projections  for  1985 
(projections  made  in  mid-1982)47  show  the  following  fill  of  authorized 
billets  by  career  personnel  (beyond  four  years  of  service)  in  the  combat 
system  technician  ratings: 

a.  Aviation:  84X  fill  of  E-5/E-6;  87X  fill  of  E-7  through  E-9. 

b.  Nonaviation:  69X  fill  of  E-5/E-6;  90X  fill  of  E-7  through  E-9. 

One  reason  why  these  shortages  are  especially  pronounced  in  the  technical 
ratings  is  that  they  possess  a  much  higher  petty  officer  content  (percent  of 
billets  authorized  in  the  E-4  to  E-9  grades)  than  the  Navy-wide  average.  The 
latter  is  currently  62  percent  (E-4  to  E-9)  or  43  percent  (E-5  to  E-9);  for 


45Congressional  Hearings  on  FY82  DoD  Appropriations,  Senate  Armed 
Services  Committee,  February-March  1981. 


46Ibid. 

47Source  data  from  Chief  of  Naval  Operations  (0P-112C),  Projections 
based  on  Force  Analysis  Simulation  Technique  (FAST)  run  in  mid-1982. 
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a.  The  much  discussed  decline  in  average  Scholastic  Aptitude  Test  (SAT) 
score  (from  1963  to  1980,  a  decline  of  11.3  percent  on  verbal  reasoning,  7.2 
percent  on  mathematical  reasoning),  camouflages  a  much  more  serious  decline  in 
above-average  students  (those  scoring  above  600  on  the  200-800  scale),  a 
serious  increase  in  underachievers  (those  scoring  below  400). 43 

b.  Over  50  percent  of  high  school  seniors  are  grossly  illiterate  or 
semi-illiterate,  unable  to  comprehend  written  material  sufficiently  to 
function  in  today's  society. 42 

c.  In  recent  years,  the  decline  in  above-average  students  has  continued 
unabated  due  to  an  instructional  focus  on  the  underachievers.  The  long-term 
trend  points  to  average  mediocrity. 

The  results  are  observable  in  Navy  recruit  accessions  as  follows: 

a.  Over  30  percent  of  high  school  graduate  accessions  (46  percent  of 
non-high  school  graduate  accessions)  have  a  reading  grade  level  below  9.0, 
which  is  four  grades  below  standard.43  (Reading  grade  levels  are 
standardized  on  the  basis  of  70  percent  of  students  being  able  to  read  and 
comprehend  50  percent  of  reading  material  in  the  next  grade;  i.e.,  high  school 
graduates  are  supposed  to  have  a  reading  grade  level  of  13.0  based  on  that 
standard) . 


b.  While  recruit  classification  tends  to  keep  most  illiterates  out  of 
"A"  Schools,  a  large  percentage  of  students  enrolled  in  "A"  Schools  report 
serious  difficulty  with  reading  course  materials  (36  percent)  or  understanding 
what  they  are  supposed  to  read  (56  percent).44 

While  the  Navy  has  initiated  several  programs  to  counter  these  trends  of 
functional  illiteracy  —  i.e.,  Academic  Remedial  Training  and  Job-Oriented 
Basic  Skills  --  there  are  limits  to  what  can  be  done  within  available  training 
budgets . 


41College  Entrance  Examination  Board,  Advisory  Panel  on  the  Scholastic 
Aptitude  Test  Score  Decline,  On  Further  Examination.  New  York,  1977. 

42Barbara  Lerner,  "American  Education:  How  Are  We  Doing?"  The  Public 
Interest.  No.  69,  Fall  1982. 

43John  J.  Mathews,  Lonnie  D.  Valentine,  Jr,,  and  Wayne  S.  Sellman, 
Prediction  of  Reading  Grade  Levels  of  Service  Applicants  from  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB).  AFHRL-TR-78-82,  Air  Force  Human  Resources 
Laboratory,  Brooks  Air  Force  Base,  TX,  December  1978. 

44Thomas  G.  Sticht,  Lynn  C.  Fox,  Robert  H.  Hanke,  and  Diane  W.  Zapf, 
Integrated  Job  Skills  and  Reading  Skills  Training  System.  NPRDC  TR-77-41,  Navy 
Personnel  Research  and  Development  Center,  San  Diego,  CA,  September  1977. 
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found  by  Dale  and  Gilroy). 39  Thus,  a  drop  from  10. A  percent  unemployment  in 
1982  to  the  five  percent  level  in  the  early  1990s  would  reduce  male  HSDG 
enlistments  in  the  Navy  by  roughly  23,500  from  the  1982  base,  everything  else 
being  equal  (constant  roilitary/civilian  pay  ratio).  As  a  result,  the  Navy 
would  fall  below  the  minimum  Congressional  standard  of  65  percent  HSDG 
accessions,  and  well  below  its  own  objective.  A  more  meaningful  measure  of 
quality,  however,  is  the  percent  of  accessions  (enlistment  contracts)  which 
are  HSDG  and  score  above  average  on  the  AFQT  (MC  I  -  IIIA).  That  has  been, 
traditionally,  about  50  percent  of  HSDG  accessions,  with  the  exception  of 
FY82 .  The  reason  why  this  may  be  a  more  meaningful  measure  of  quality  is 
explained  next. 

Educational  Attainment  Has  Lost  Much  of  its  Meaning  as  a  Quality  Indicator. 

The  two  standard  indicators  of  quality  used  by  DoD  are  high  school  diploma  and 
AFQT  percentile  score;  all  quality  standards  are  expressed  in  terms  of  these 
two  indicators,  but  not  in  terms  of  the  two  combined  (except  for  the  bottom 
group:  MC  IV  is  eligible  only  with  a  high  school  diploma).  Graduation  from 
high  school  is  an  important  indicator  of  suitability  for  military  service.  It 
is  related  to  personality  traits  (achievement  motivation  and  self-discipline) 
that  are  desirable  in  military  service.  Statistical  analysis  shows  that 
first-term  attrition  of  non-high  school  graduates  is  about  double  that  of  high 
school  graduates;  disciplinary  problems  are  also  more  prevalent  among  non-high 
school  graduates.40  But  as  an  indicator  of  educational  achievement,  high 
school  graduation  has  lost  much  of  its  value  due  to  the  long-term  decline  in 
quality  of  education  in  public  high  schools.  A  few  statistics  follow: 


39A  technical  issue  is  that  the  dependent  variable  used  (enlistment 
rate  instead  of  number  of  enlistment  contracts)  implicitly  assumes  the 
elasticity  of  the  eligible  quality  pool  to  be  1.0.  Previous  studies  have 
found  a  much  lower  supply  elasticity;  i.e.,  the  effective  supply  of  high 
school  graduates  shows  a  less  than  1-for-l  decline  with  a  declining  number  of 
high  school  seniors.  The  argument  is  that  only  when  the  entire  pool 
susceptible  to  enlistment  is  contacted  effectively  by  recruiters,  can 
enlistments  be  expected  to  decline  proportionately  with  the  eligible  pool.  To 
the  extent  this  is  true,  Dale  and  Gilroy  overestimate  the  unemployment 
effect.  Because  an  increasing  portion  of  public  high  schools  is  adopting  the 
Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  for  counseling  purposes,  it 
is  believed  that  the  impact  of  this  supply  elasticity  effect  may  be  much  less 
today  than  it  was  in  the  mid-1970s.  It  will  become  even  less  so  in  the 
future,  with  every  recruiter  in  hot  pursuit  of  every  male  high  school  graduate 
available  (see  previous  observation). 


Washington,  DC,  p.  II-3,  October  1982. 
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The  difficulty  of  sustaining  the  Navy's  qualitative  goal  is  illustrated 
by  the  recruiting  results  in  FY82.  This  was  a  year  with  impressive  recruiting 
results,  helped  along  with  a  peak  supply  of  18  year  olds,  high  youth 
unemployment,  reduced  recruiting  quotas,  and  a  substantial  recruiting  effort. 
Yet,  the  actual  percent  of  HSDG  non-prior-service  accessions  achieved  was  74 
percent  vice  the  82  percent  goal.  The  number  of  upper-mental  category  (MC  I  - 
IIIA)  male  HSDG  enlistment  contracts,  however,  was  43,861,  the  first  time  in 
history  that  this  group  exceeded  50  percent  of  accessions.36 

The  Effect  of  Unemployment  has  been  Underestimated  in  the  Past.  Previous 
studies  show  a  bewildering  range  in  estimates  of  the  sensitivity  of  the  number 
of  quality  accessions  to  changes  in  independent  variables,  both  exogenous 
(pay,  unemployment)  and  endogenous  (marketing  effort,  numbers  of 
recruiters).3'  The  sensitivity  (elasticity,  in  econometric  jargon)  to 
unemployment,  however,  appears  to  have  been  underestimated  in  most  studies, 
due  to  inadequate  data,  improper  statistical  methods,  or  an  improper 
model.38  There  is  a  consensus  among  the  research  community  that  one  model 
cannot  be  applied  indiscriminately  to  all  accessions:  high-quality  recruits 
are  supply-limited,  low-quality  are  demand-limited;  i.e.,  the  two  categories 
require  different  models.  The  recent  results  obtained  by  Dale  and  Gilroy  show 
an  unemployment  elasticity  of  0.94,  relating  the  enlistment  rate  of  male  high 
school  graduates  to  male  teenage  unemployment.  (The  rate  equals  the  total 
enlistment  contracts —  i.e.,  accessions  plus  delayed  entry  program  —  divided 
by  the  male  youth  population  of  16  to  19  year  olds.)  Converting  to  total 
unemployment  rate  and  applying  this  to  the  Navy,  results  in  an  estimated  drop 
of  6.8  percent  (4,300)  male,  non-prior-service,  HSDG  enlistments  per  one 
percent  drop  in  the  national  unemployment  rate.  This  estimate  of  the 
unemployment  effect  is  about  triple  that  reported  by  many  previous  studies 
(with  the  exception  of  one  which  found  an  unemployment  effect  2  1/2  times  that 


36This  information  is  based  on  LHl's  review  of  actual  recruiting  data 
during  FY82. 

37For  a  recent  comprehensive  survey,  see:  Richard  C.  Morey  and  John  N. 
McCann,  "Armed  Services  Recruiting  Research:  Issues,  Findings,  and  Needs," 
Naval  Research  Logistics  Quarterly,  Vol.  30,  (1983),  pp.  697-719. 

38Charle8  Dale  and  Curtis  Gilroy  (Personnel  Policy  Research  Group,  U.S. 
Army  Research  Institute),  "The  Effects  of  the  Business  Cycle  on  the  Size  and 
Composition  of  the  U.S.  Army,"  Atlantic  Economic  Journal.  Vol.  11,  No.  1 
(March  1983). 
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college;  and  with  the  overcapacity  faced  by  colleges  in  the  late  1980s,  the 
percent  going  on  to  college  can  be  expected  to  increase  further.  Propensity 
to  enlist  among  college-bound  high  school  graduates  or  those  in  college  is 
much  lower  than  for  those  who  are  undecided. 

The  third  factor  is  the  long-term  trend  in  the  civilian  labor  market. 

With  the  high  youth  unemployment  of  the  early  1980s  beginning  to  recede,  the 
number  of  jobs  in  the  national  economy  is  projected  to  grow  by  22  percent 
between  1982  and  1990  (Department  of  Labor  projection,  1982),  yet  the  work, 
force  (total  U.S.  population,  18  to  64  years  of  age)  will  grow  only  by  ten 
percent. The  increasing  scarcity  of  youthful  workers  (the  18  to  24  age 
group  declines  by  15  percent  from  1982  through  1990)  will  ensure  ample  work 
opportunities  for  high  school  graduates,  especially  those  of  above  average 
quality. 

The  combined  result  of  these  three  factors  is  that  current  quality 
standards  will  probably  not  be  sustainable.  For  example,  the  first  two 
factors  alone  imply  that  seven  out  of  ten  uncommitted  (noncollege-bound)  male 
high  school  graduates  must  be  enlisted  in  1992  just  to  meet  the  75  percent 
standard.  This  is  nearly  double  the  ratio  in  1981,  when  four  out  of  ten  were 
required. 3^  This  implies  at  least  a  doubling  of  the  recruiting  effort  and 
the  necessity  to  contact  every  individual  male  high  school  graduate.  The 
Center  for  Naval  Analyses  model  of  high  school  graduate  enlistments  projects  a 
chronic  shortfall  in  quality  accessions  for  the  Navy  by  1989.35  Even  under 
the  most  favorable  circumstances  (military/civilian  pay  ratio  remaining 
constant,  no  effects  from  declining  unemployment  rates),  the  model  projects  a 
shortfall  of  16  to  26  percent  in  HSDG  accessions  (the  extent  of  the  shortfall 
depends  on  military  pay)  compared  to  the  Navy's  objective  (which  would  be 
59,900  based  on  a  goal  of  82  percent  non-prior-service  male  accessions).  The 
two  other  factors  (more  people  college-bound  and  more  civilian  employment 
opportunities)  would  worsen  this  shortfall. 


33Bureau  of  the  Census,  Population  Estimates  and  Projections,  Series 
P-25,  No.  922,  October  1982. 

34Richard  W.  Hunter  and  Gary  R.  Nelson,  "Eight  Tears  with  the 
All-Volunteer  Armed  Forces."  Paper  included  in  Lt.  Gen.  Brent  Scowcroft,  USAF 
(Ret.),  Editor.  Military  Service  in  the  United  States.  The  American 
Assembly,  Columbia  University,  1982. 

35john  A.  Cirie,  James  J.  Miller,  and  H.  Wallace  Sinaiko,  "Department 
of  Defense  and  Navy  Personnel  Supply  Models,"  Proceedings  of  Workshop  convened 
at  request  of  Navy  Secretariat,  conducted  by  the  Smithsonian  Institute  and  the 
Office  of  Naval  Research,  May  1981. 
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19.5  percent  (from  1982  to  1988)  as  compared  to  the  Navy-wide  growth  in 
enlisted  requirements  of  8.6  percent. 28  This  change  in  composition  is  even 
more  pronounced  in  the  Army.  As  a  result,  quality  standards  for  new 
accessions  have  increased  DoD-wide: 

a.  Mental  Category  (MC)  IV  (Armed  Forces  Qualification  Test  (AFQT) 
percentile  10  to  30)  candidates  without  high  school  diplomas  are  ineligible. 

b.  High  School  Diploma  Graduates  (HSDG)  scoring  in  the  bottom  quarter 
of  MC  IV  (AFQT  interval  10-15)  are  ineligible.  (The  Services  set  their  own 
standards  above  this  Congressionally  mandated  minimum;  the  Navy's  cut-off  is 
currently  an  AFQT  percentile  score  of  17). 

c.  The  maximum  percentage  of  MC  IV  accessions  is  20  percent  (down  from 
the  25  percent  standard  in  FY82). 

d.  The  threshold  for  the  percentage  of  male  non-prior-service  HSDG 
accessions  has  been  increased  from  65  percent  (current  Congressional  standard) 
to  75  percent  (Defense  Guidance  1985-1989).  (The  Services  set  their  own  goals 
based  on  annual  reviews.  The  Navy  Recruiting  Command,  in  1982,  increased  its 
objective  from  76  to  82  percent  HSDG).29 

The  second  factor  is  the  long-term  change  in  the  composition  of  supply 
(military  eligible  pool).  The  overall  decline  of  the  youth  cohort  has  a 
proportionate  effect  on  the  number  of  high  school  graduates;  i.e.,  a  decline 
of  24.5  Dercent  from  FY82  to  FY92,  if  the  current  dropout  rate  of  23 
percent3^  remains  constant  (weighted  average  of  20  percent  for  whites,  36 
percent  for  blacks,  40  percent  for  Hispanics) .31  But  the  youth  cohort 
decline  is  concentrated  in  whites;  black  high  school  enrollments  will  actually 
show  a  small  increase.  Another  trend  is  that  an  increasing  proportion  is 
college-bound.  Traditionally,  about  50  percent  attend  college;  but  a  1982 
survey32  shows  that  56  percent  of  male  high  school  graduates  continue  on  to 


28Chief  of  Naval  Operations  (0P-112D),  NAMPS  run  of  28  April  1982. 

29CDR  Donald  L.  Pilling  (USN).  "The  Dwindling  Muster,"  U.S.  Naval 
Institute  Proceedings,  7  June  1982. 

30Bureau  of  Labor  Statistics,  Special  Labor  Force  Reports;  data  as  of 
March  1981. 

33Caspar  W.  Weinberger  (Chairman),  Military  Manpower  Task  Force  Report 
to  the  President  on  the  Status  and  Prospects  of  the  All  Volunteer  Force. 
Washington,  DC,  October  1982. 

32Jerald  G.  Bachman,  American  High  School  Seniors  View  the  Military: 
1976-1982.  Institute  for  Social  Research,  University  of  Michigan,  Ann  Arbor, 
HI,  November  1982. 
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Figure  3.  Trends  in  Male  Youth  Population. 


the  above  ratio  will  have  declined  to  3.5  by  1990  and  reach  a  low  of  3.3  in 
1992;  i.e.,  one  out  of  every  three  qualified  males  must  be  signed  up.27 

The  Demand/Supply  Relationship  for  High-Quality  Accessions  Will  Become  Tighter 
Yet .  Three  major  factors  are  responsible  for  making  the  competition  for 
high-quality  youth  very  tight.  First,  the  composition  of  military  demands  is 
changing.  The  growth  of  the  Navy  is  concentrated  in  the  highly  technical 
ratings;  for  example,  the  combat  system  technicians  are  expected  to  grow  by 


27Richard  W.  Hunter  and  Gary  R.  Nelson,  "Eight  Years  with  the 
All-Volunteer  Armed  Forces."  Paper  included  in  Lt.  Gen.  Brent  Scowcroft,  USAF 
(Ret.),  Editor.  Military  Service  in  the  United  States.  The  American 
Assembly,  Columbia  University,  1982. 
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Figure  2.  Planned  Elimination  of  Petty  Officer  Shortages. 

SOURCE:  Caspar  W.  Weinberger  (Chairman),  Military  Manpower  lash  Force 

Report  to  the  President  on  the  Status  and  Prospects  of  the  All 
Volunteer  Force.  Washington,  DC,  October  1982. 


The  Demand/Supply  Relationship  Will  Become  Tight.  The  commonly  used  measure 
of  tightness  for  military  recruiting  is  the  number  of  qualified,  18  year  old 
males  available  per  enlistment  contract.  In  the  1950s  and  1960s,  this  ratio 
hovered  around  1.5  (i.e.,  the  Services  needed  two  out  of  every  three  qualified 
males),  so  that  a  military  draft  was  necessary.  With  the  post-Vietnam  draw 
down  and  changing  demographics,  this  ratio  increased  rapidly  in  the  early 
19708,  eaching  a  current  level  of  about  4.5.  This  made  the  All-Volunteer 
Force  possible.  Even  with  the  programmed  DoD-wide  accession  requirements  of 
330,000  to  340,000  per  year  throughout  the  remainder  of  the  1980s2*  (a 
number  which  may  turn  out  to  be  too  low  due  to  lower  than  planned  retention), 


2*Caspar  W.  Weinberger  (Chairman),  Military  Manpower  Task.  Force  Report 
to  the  President  on  the  Status  and  Prospects  of  the  All  Volunteer  Force. 
Washington,  DC,  October  1982. 
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Wavy  Manpower  Plans  Assume  No  Increase  in  Accessions.  Formal  plans  are  to 
accommodate  the  growth  in  manpower  through  increased  retention  so  that 
non-prior-service  accession  requirements  remain  relatively  stable  throughout 
the  remainder  of  the  1980s.  Programmed  non-prior-service  (enlisted)  accession 
requirements  range  from  a  high  of  86,000  (FY85)  to  a  low  of  82,000  (FY88). 

(It  was  98,000  in  FY80  and  92,000  in  FY81.)20  Increased  retention  is  also 
counted  on  to  eliminate  the  existing  petty  officer  shortage  (see  Figure  2). 
Favorable  experience  in  the  past  two  years,  especially  through  lateral  entry 
of  prior-service  personnel,  agreed  with  this  plan.  For  example,  the  petty 
officer  shortage  declined  from  22,000  to  18,000  by  the  end  of  FY82. 

Manpower  Supply  is  Decreasing.  The  demographic  trends  are  well-known.  The 
market  of  primary  interest  to  military  recruiters,  males  in  the  age  group  of 
17  to  21  years  old,  was  10.784  million  in  1982  (year  mid-point).2*  it  will 
decline  15  percent  by  1990,  and  22.5  percent  by  1995, 22  which  is  the  trough 
for  this  age  group  (see  Figure  3).  Uithin  this  group,  the  prime  recruiting 
cohort  (males  reaching  18  years  of  age)  shows  a  drop  of  24.5  percent  from  1982 
to  1992,  the  trough  for  18  year  olds.  After  1995,  male  youths  in  the  17  to  21 
years  age  bracket  will  slowly  increase;  this  group  will  stay  below  the  ten 
million  level  and,  according  to  Bureau  of  the  Census  projections,  will 
stabilize  by  the  middle  of  the  next  century  at  a  level  13.5  percent  below  that 
of  1982.  While  the  pending  decline  still  leaves  a  large  number  available,  a 
significant  portion  does  not  count  towards  the  supply  of  potential  candidates 
for  military  service.  The  estimates  of  the.  percentage  of  the  age  group  not 
qualifying  for  military  service  (based  on  current  mandatory  minimum  physical, 
moral,  and  mental  standards)  range  from  22  percent  overall23  to  27  percent 
for  whites,  37  percent  for  blacks24  to  a  weighted  average  of  34  percent 
projected  for  the  late  1980s25. 


20Ibid. 

21Bureau  of  the  Census,  Population  Estimates  and  Projections,  Series 
P-25,  No.  922,  October  1982. 

22Ibid. 

23Department  of  Defense,  Soviet  Military  Power  (Washington,  DC: 
Government  Printing  Office,  September  1981  (A  1983  update  was  published  in 
Armed  Forces  Journal  International.  March  1983). 

24Thomas  S.  Gates  (Chairman),  The  Report  of  the  President's  Commission 
on  an  All  Volunteer  Force,  Washington,  DC,  February  1970. 

25CDR  Donald  L.  Pilling  (USN).  "The  Dwindling  Master,"  U.S.  Naval 
Institute  Proceedings  (June  1982),  pp.  35-39. 
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TABLE  4.  NAVY  RATINGS  INVOLVING  ELECTRONIC  MAINTENANCE  SKILLS 


RatinK 

Occupational  Title 

Open  to 
Women 

Number  of2 
Unique  NECs 

AQ 

Aviation  Fire  Control 

40/60 

Yes 

37 

Technician 

AT 

Aviation  Electronics 

60/40 

Yes 

73 

Technician 

AX 

Aviation  Antisubmarine 

40/60 

Yes 

14 

Warfare  Technician 

CTM 

Cryptologic  Technician, 

60/40 

Yes 

35 

Maintenance 

DS 

Data  Systems  Technician 

60/40 

Yes 

22 

ET 

Electronics  Technician 

60/40 

Yes 

78 

EU 

Electronics  Warfare 

60/40 

No 

14 

Technician 

FTG 

Fire  Control  Technician 

60/40 

Yes 

(Gun  Fire  Control) 

\  59 

((FT  Total) 

FTM 

Fire  Control  Technician 

60/40 

No 

/ 

(Missile  Fire  Control) 

GMH 

Gunner's  Mate  (Missiles) 

60/40 

No 

16 

(Missile  Fire  Control) 

(GM  Total) 

STG 

Sonar  Technician  (Surface) 

60/40 

No 

5  41 

STS 

Sonar  Technician  (Submarine) 

60/40 

No 

)(ST  Total) 

TM 

Torpedoman's  Mate 

60/40 

Yes 

14 

Approximate  split  of  authorized  billets  between  Fleet  and  shore 
establishment  per  Navy  Enlisted  Career  Guide  1980-1981. 

2NECs  established  for  each  Bource  rating  by  NAVPERS  18068D,  Navy  Enlisted 
Manpower  and  Personnel  Classifications  and  Occupational  Standards,  April  1980 
issue. 

NOTE:  Excludes:  NT  (Missile  Technician)  and  FTB  (Ballistic  Missile  Fire 
Control  Technician),  both  supporting  strategic  systems;  and  TD  (Training  Device 
Technician),  being  disestablished. 


33 


NAVTRAEQUIPCEN  KDA903-81-C-0166-1 


SECTION  III 

IMPACT  OF  FUTURE  TRENDS 


Two  of  the  factors  affecting  maintenance  performance,  which  the  Navy  can 
influence,  but  cannot  completely  control,  are  personnel  availability  (both  in 
a  quantitative  and  qualitative  sense)  and  the  future  direction  of  technology 
as  applied  in  combat  systems.  This  study  was  focused,  therefore,  on 
projecting  these  two  factors  to  the  mid-1990s,  and  assessed  the  resulting 
implications  in  terms  of  maintenance  skill  requirements  and  maintenance 
performance. 

MANPOWER  AND  PERSONNEL 

INTRODUCTION.  Manpower  and  personnel,  in  combination,  refer  to  the  ability  of 
each  Service  to  meet  and  sustain  its  requirements  (determined  by  force 
structure)  with  available  personnel  (determined  by  the  inventory,  attrition 
and  retention  rates,  and  extent  to  which  new  accession  requirements  can  be  met 
through  recruiting),  both  in  terms  of  quantity  and  quality.  Because  this  is 
the  single  most  critical  issue  for  the  continued  success  of  the  All-Volunteer 
Force,  it  receives  a  great  deal  of  attention.  Massive  amounts  of  data  are 
routinely  collected  and  anaylzed,  both  within  and  outside  the  Services,  for 
use  in  manpower  planning,  personnel  management,  and  policy  decisions.  Some  of 
the  data  pertaining  to  the  U.S.  Navy  was  examined  and  is  focused  on  the 
ratings  responsible  for  non-strategic  combat  systems  maintenance  (see  Table  4) 

OBSERVATIONS . 

Manpower  Requirements  are  Increasing.  With  the  end  of  FY82  as  a  baseline  and 
using  last  year's  planning  data  for  FY88,  the  increase  in  Navy  manpower  is  as 
follows : 


a.  Unconstrained  Enlisted  Requirements:  8.6X  growth.18 

b.  Programmed  Active  Duty  Strength  (officers  and  enlisted):  12. 7X 
growth . 1^ 

For  subsequent  years,  Navy  manpower  plans  show  an  average  growth  of  one 
percent  per  year  into  the  1990s. 


18Chief  of  Naval  Operations  (OP-112D),  NAMPS  run  of  28  April  1982. 

^Caspar  W.  Weinberger  (Chairman),  Military  Manpower  Task  Force  Report 
to  the  President  on  the  Status  and  Prospects  of  the  All  Volunteer  Force. 
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d.  Leaving  the  problem  unsolved  is  not  cost-effective.  First,  the  cost 
implications  of  poor  maintenance  performance  are  staggering,  but  not  traceable 
as  such,  by  current  accounting  systems.  Second,  the  increased  reliance  on 
Fleet  support  to  meet  readiness  requirements  is  at  the  expense  of  skill 
stagnation  of  Fleet  technicians. 

POSTSCRIPT 

The  assessment  of  combat  systems  maintenance  performance,  based  on 
empirical  and  substantive  data,  is  grim.  It  must  be  added  that  this 
assessment  pertains  to  the  generation  of  combat  systems  currently  in  the 
Fleet;  i.e.,  designed  in  the  1960s  or  early  1970s  when  design  for 
maintainability  and  testability  received  much  less  emphasis  than  it  does 
today.  The  very  recent  changes  in  acquisition  policies  and  the  evolution  of 
testability  as  a  true  engineering  discipline  (as  will  be  discussed  in  Section 
IV)  have  potential  for  reducing  the  extent  of  the  problem  with  the  next 
generation  of  combat  systems.  The  current  systems,  however,  will  remain  with 
the  Fleet  well  into  the  1990s,  so  that  the  problem  will  not  just  disappear 
within  the  next  few  years.  Furthermore,  an  overly  optimistic  expectation  of 
improvements  with  the  next  generation  in  the  1990s  may  not  be  warranted  as  a 
result  of  the  future  trends  described  next.  In  other  words,  maintenance  skill 
training  is  the  most  crucial  issue  today,  and  will  remain  so  in  the 
foreseeable  future. 
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as  assistance  with  equipment  maintenance  problems.  Once  the  battle  group 
arrives  in  Pearl  Harbor,  the  units  go  into  a  repair  mode.  HOTU  5  personnel, 
supplemented  by  MOTU  1  technicians  (homeported  in  Pearl  Harbor),  assist  in 
accomplishing  repairs.  At  the  end  of  the  72-hour  stay  in  Pearl  Harbor,  MOTU  5 
personnel,  to  the  extent  they  are  not  longer  needed,  return  to  San  Diego. 
Normally,  about  50  percent  (11  technicians)  stay  with  the  battle  group.  The 
battle  group  continues  its  transit  to  Subic  Bay,  Philippine  Islands 
(approximately  four  weeks).  The  MOTU  5  technicians  continue  to  perform  the 
same  functions  as  on  the  first  leg  of  the  journey.  Once  in  Subic  Bay,  units 
go  again  into  a  repair  mode  lasting  five  days.  MOTU  5  personnel  are  now 
supplemented  by  personnel  from  MOTU  7  to  assist  ship's  force  in  combat  system 
repairs.  Once  again,  MOTU  5  personnel  no  longer  needed  return  to  San  Diego; 
some  MOTU  5  technicians,  however,  stay  with  the  battle  group  for  the  entire 
deployment  (seven  months)  to  compensate  for  shortfalls  in  on-board  expertise. 

This  battle  group  technical  support  program  was  initiated  in  1980  due  to 
the  low  combat  systems'  A0.  The  reason  the  program  continues  today  is  the 
continued  shortfall  in  the  level  of  skills  and  knowledge  of  the  technicians, 
causing  a  steadily  increasing  dependence  on  MOTU  5  for  training  and  technical 
assistance.  MOTU  5  is  the  largest  Fleet  support  activity  in  the  Navy. 

SUMMARY  FLEET  ASSESSMENT.  The  two  Fleet  programs  described  above  are  strictly 
viewed  as  necessary  "band-aid"  programs.  If  technical  training  continues  in 
its  present  form,  however,  the  resulting  void  in  technical  proficiency  will 
cause  these  programs  to  become  permanent  fixtures.  In  a  peacetime  Navy,  where 
schedules  are  known  in  advance,  these  programs  may  flourish;  in  wartime,  there 
are  not  enough  assets  to  support  the  entire  Fleet  this  way. 

CONCLUSIONS 

The  evidence  presented  in  this  chapter  yields  the  following  conclusions: 

a.  Inadequate  maintenance  performance  is  a  serious  problem.  It 
inhibits  the  Navy  from  achieving  realistic  readiness  goals  in  peacetime  and 
bodes  ill  for  the  Navy's  ability  to  sustain  wartime  mission  requirements. 

b.  Inadequate  maintenance  performance  is  a  systemic  problem,  influenced 
by  a  multitude  of  interrelated  factors.  Maintenance  performance  will  improve 
through  better  maintenance  training  and  aiding  only  if  improvements  are  made 
in  other  factors  such  as  job  design,  human  factors  engineering,  training 
philosophy,  design  for  maintainability,  etc. 

c.  The  Navy  is  addressing  the  symptoms,  not  the  causes,  of  the 

problem.  The  Systems  Commands  focus  on  realiability  improvements  and  supply 

support.  The  Fleet  is  doing  the  best  it  can,  steadily  increasing  its 

dependence  on  Fleet  support  from  the  shore  establishment  both  for  combat 

system  maintenance  (preventive  as  well  as  corrective)  and  practical  job 
training.  The  Fleet's  ability  to  continue  doing  so  is,  however,  limited  by 
Fleet  support  billets  and  funding;  in  wartime,  it  would  be  impossible. 
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In  order  to  assess  and  monitor  the  contributions  made  by  the  Mobile 
Training  Team,  COMNAVSURFLANT  plans  to  use  two  current  combat  system 
examinations  as  measures  of  effectiveness:  the  Combat  Systems  Post  Overhaul 
Examination  (conducted  shortly  after  overhaul)  and  the  Combat  Systems 
Operational  Readiness  Examination  (conducted  upon  completion  of  refresher 
training).  Although  training  is  not  the  only  factor  measured  by  these 
examinations,  it  is  the  most  important  aspect  according  to  COMNAVSURFLANT. 

PACIFIC  FLEET  PROGRAM.  Commander,  Naval  Surface  Force,  U.S.  Pacific  Fleet 
(COMNAVSURFPAC) ,  does  not  have  a  formalized  program  similar  to  the  CSIP 
created  by  COMNAVSURFLANT.  However,  Logistics  Command,  Pacific  (LOGPAC) 
provides  a  tremendous  amount  of  technical  training  and  support  to  Vest  Coast 
units.  LOGPAC  is  a  Fleet  support  command  peculiar  to  the  Pacific  Fleet;  it 
represents  a  consolidation  of  diverse  activities  which,  for  the  Atlantic 
Fleet,  are  dispersed  among  the  Readiness  Support  Group,  MOTUs,  Systems  Command 
field  activities,  and  others.  Whereas  the  CSIP  is  keyed  to  an  individual 
ship's  overhaul  and  post-overhaul  cycle,  the  Pacific  Fleet's  program  (which 
has  no  formal  name)  is  keyed  to  the  battle  group  deployment  cycle.  The 
primary  thrust  of  this  program  is,  again,  technical  training. 

A  deploying  battle  group  is  assembled  approximately  four  to  five  months 
prior  to  deployment.  As  part  of  the  predeployment  work  up,  the  battle  group 
conducts  a  readiness  exercise  approximately  three  months  prior  to  deployment. 
Just  prior  to  the  exercise,  MOTU  5  (homeported  in  San  Diego)  provides  the 
first  step  in  the  training  process.  Along  with  members  of  the  deploying  Group 
Commander's  staff,  MOTU  5  assembles  the  combat  systems  officers  from 
individual  units  within  the  battle  group  and  reviews  the  status  of  their 
combat  systems'  equipment  and  maintenance  technician  training.  MOTU  5  then 
schedules  and  provides  as  much  technical  training  assistance  as  feasible  prior 
to  commencement  of  the  exercise.  As  a  result,  units  go  to  sea  with  a  higher 
readiness  posture  than  in  the  past  and,  therefore,  can  participate  more 
realistically  in  the  exercise.  All  necessary  training  cannot  be  completed  in 
the  available  time,  however,  so  that  units  may  not  have  the  on-board  expertise 
to  repair  combat  system  casualties  occurring  at  sea  during  the  exercise.  In 
order  to  assist  in  this  area,  MOTU  5  provides  six  to  eight  of  its  technicians 
to  the  battle  group  for  the  duration  of  the  exercise.  The  fields  of  expertise 
for  these  technicians  are  Naval  Tactical  Data  System,  communications  (HF,  UHF, 
and  LINK),  and  radar  (2D,  3D,  and  fire  control).  During  the  exercise,  usually 
three  to  five  weeks  in  duration,  these  technicians  visit  battle  group  elements 
that  need  assistance  in  accomplishing  technical  training  or  correcting 
equipment  casualties. 

Upon  completion  of  the  exercise,  units  begin  preparing  for  deployment, 
usually  with  assistance  from  the  Fleet  support  community,  concentrating  on 
repair  and  calibration  of  combat  system  equipments  in  order  to  deploy  at  the 
highest  level  of  operational  readiness  attainable.  When  the  battle  group 
finally  deploys,  MOTU  5,  by  standard  operating  procedure,  assigns 
approximately  22  combat  systems  maintenance  technicians  to  make  the  transit 
with  the  battle  group  to  Pearl  Harbor,  Hawaii,  a  period  of  two  weeks.  These 
technicians  provide  technical  training  to  units  in  the  battle  group,  as  well 
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the  combat  system  technician  ratings  as  a  group,  this  parameter  is  about  88 
percent  (E-4  or  E-9)  or  64  percent  (E-5  to  E-9).48  As  a  result  of  the  petty 
officer  shortfall,  the  experience  profile  (years-in-service  distribution)  of 
personnel  in  each  of  these  technical  ratings  is  heavily  biased;  in  1982,  the 
average  months  in  service  of  personnel  in  these  ratings  ranged  from  a  low  of 
66  to  a  high  of  85  compared  to  the  Navy-wide  average  of  67  months.4®  Host 
personnel  (57  to  71  percent,  depending  on  rating)  have  six  or  less  years  in 
service.50  This  profile  of  inexperience  will  become  worse  with  the  planned 
growth  in  these  ratings  at  a  rate  more  than  double  the  Navy-wide  rate. 

ASSESSMENT.  The  situation  is  not  hopeless.  With  appropriate  recruiting 
efforts  and  us?  of  available  incentives,  the  Navy  should  be  able  to  meet  its 
accession  requirements  into  the  1990s,  if  military  pay  comparability  is 
maintained.  Maintaining  the  modest  quality  standards  currently  in  effect,  let 
alone  achieving  current  objectives,  however,  will  be  impossible  without  major 
changes  in  policy.  Similarly,  with  respect  to  retention,  if  the  unemployment 
rate  comes  down  sufficiently  and  confidence  is  gained  in  civilian  employment 
prospects,  retention  can  be  anticipated  to  decline  to  traditional  levels.  The 
Navy's  manpower  plan  is  fine  as  a  goal;  as  a  plan  it  appears  to  be 
unrealistic,  unless  major  changes  in  policy  are  implemented  to  increase 
lateral  entry  of  prior-service  personnel  and  increase  retention.  Without 
those  changes,  a  continued  shortage  in  the  senior  enlisted  grades  (E-5  through 
E-9)  in  the  combat  system  technician  ratings  can  be  anticipated,  compensated 
in  quantity  (but  not  in  quality)  by  an  overfill  in  the  junior  grades. 

IMPLICATIONS  FOR  MAINTENANCE.  The  net  effect  of  the  above  trends  is  that  less 
experienced  personnel  must  carry  more  of  the  maintenance  burden. 

Specifically,  first-term  technicians  will  be  required  to  perform  a  much  larger 
portion  of  corrective  maintenance  tasks  than  currently.  Improvements  in 
technical  training  and  maintenance  job  aids  will  be  essential  to  make  this 
possible,  unless  maintenance  skill  requirements  decline  as  a  result  of  future 
combat  system  technology. 

MAINTENANCE  TRENDS  OF  CURRENT  TECHNOLOGY 

The  maintainability  characteristics  and  maintenance  concepts  of  three 
"emerging"  combat  systems,  one  in  each  Navy  community,  were  examined  in  order 
to  identify  changes  from  the  current  baseline  (examples  of  currently 
operational  equipment  in  the  Fleet  as  summarized  in  Section  II)  and  to  assess 
future  maintenance  trends.  Two  of  these  systems,  the  AEGIS  Weapon  System  and 
F/A-18A  HORNET  avionics,  have  already  reached  operational  status;  the  third. 


48Defense  Manpower  Data  Center,  Requirements  and  Authorizations  for 
FY82,  Inventory  as  of  30  June  1982. 


49Ibid. 
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the  Submarine  Advanced  Combat  System  (SUBACS),  is  still  under  development  with 
a  design  freeze  planned  in  1989.  The  observations  made  are  summarized  here  to 
illustrate  some  fundamental  differences  in  future  directions  pursued  in  each 
community. 

AEGIS.  AEGIS  sets  a  clear  example  for  a  much-needed  trend;  its  inherent 
complexity  is  far  above  that  of  the  comparable  system  currently  deployed  (Mk 
86  GFCS),  while  its  maintenance  complexity  is  less.  The  built-in  performance 
monitoring/fault  localization  (PM/FL)  system,  known  as  Operational  Readiness 
Test  System  (ORTS),  significantly  reduces  the  uncertainty  and  ambiguity 
associated  with  corrective  maintenance  of  the  Mk  86  GFCS.  Through  reference 
codes  displayed  on  the  ORTS  console  display,  the  technician  is  either  directed 
to  the  proper  module  replacement  procedure  (if  the  fault  has  been  isolated  by 
ORTS  to  a  single  module),  or  to  the  proper  fault-isolation  work  package  (to 
manually  isolate  the  fault  within  a  given  ambiguity  group).  The  needed 
technical  data  are  retrieved  through  an  automatic  microfiche  retrieval  system, 
next  to  the  ORTS  console  displays,  and  can  be  printed  on  the  spot.  AEGIS 
provides  a  de  facto  illustration  of  integrated  diagnostics  —  a  concept  which 
will  be  discussed  later  in  this  section.  Factors  that  may  have  contributed  to 
this  successful  implementation  of  automated/semi -automated  maintenance 
diagnostics  include  the  following: 

a.  program  management  emphasis  on  availability, 

b.  no  cutting  corners  in  maintainability, 

c.  a  long  maturation  program  prior  to  Fleet  introduction  (prototypes 
have  been  used  since  1973  for  land-based  testing,  since  1975  for  at-sea 
testing) , 

d.  professional  engineering  support  to  the  program  office,  independent 
of  the  prime  contractor, 

e.  a  closed-loop  failure  reporting,  analysis,  and  corrective  action 
system  operated  from  1973  to  the  present, 

f.  verification  of  all  "no-go"  and  "go"  paths  in  the  ORTS  software  (the 
latter  is  still  in  progress). 

AEGIS  appears  to  show  that  "it"  can  be  done.  Three  observations  must  be 
added  to  temper  unwarranted  enthusiasm.  One  is  that  the  ORTS  design  concept 
may  have  gone  too  far  in  its  system-level  orientation.  As  a  result,  the 
effectiveness  of  ORTS  is  dependent  on  the  operational  status  of  the  AEGIS 
system.  When  the  system  degrades  as  a  result  of  maintenance  deferrals  or 
inadequate  maintenance,  ORTS  fault  detection  and  isolation  capabilities 
degrade  too;  when  one  or  more  subsystems  are  "down,"  ORTS  is  ineffective  for 
the  subsystems  that  are  "up."  For  example,  ORTS  does  not  work  on  the  Fire 
Control  System  unless  the  total  weapon  system  (including  Command  and  Decision 
System,  Weapons  Control  System,  SPY-1A  Radar  System)  is  "up."  One  lesson 
learned  might  be  the  need  to  address  this  issue  in  testability 
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specifications.  Normally,  such  specifications  are  very  careful  in  requiring 
that  BIT  failures  must  not  affect  the  operational  status  or  performance  of  the 
weapon  system;  but  reverse  requirements  (i.e.,  that  BIT  performance  must  not 
be  affected  by  graceful  degradation  of  the  weapon  system  or  by  having  portions 
of  the  system  "down")  are  never  addressed,  as  far  as  can  be  determined. 

The  second  observation  pertains  to  the  danger  of  drawing  hasty 
conclusions  from  the  favorable  experience  of  the  lead  ship  of  the  CG-47  Class 
during  its  first  cruise.  It  is  manned  in  accordance  with  authorizations,  and 
all  AEGIS  technicians  have  received  special  training.  Maintenance 
supervisors,  while  satisfied  with  ORTS  capabilities,  articulated  their  concern 
about  what  will  happen  to  AEGIS  once  it  is  manned  through  the  standard  Navy 
personnel  and  training  system.  In  this  context,  it  may  be  useful  to  recall 
that  the  program  manager's  estimate  of  AEGIS  manning  requirements  was  for  1st 
and  2nd  class  petty  officers  in  the  DS  rating,  not  the  2nd  and  3rd  class  FTs 
currently  authorized. 

The  third  observation  pertains  to  the  level  of  supply  support  needed  to 
sustain  high  A0.  Current  methodologies  for  determining  shipboard  stockage 
of  spares  and  repair  parts  are  based  on  historic  demands.  Such  a  methodology 
can  no  longer  provide  adequate  support  for  a  complex  system  such  as  AEGIS: 
the  high  count  of  replaceable  items  and  their  individual  high  reliability 
combine  to  result  in  a  very  low  probability  of  any  particular  item-type  to 
fail  within  a  specific  time  period,  thereby  disqualifying  most  items  from 
being  stocked  aboard.  There  is  a  need  to  replace  traditional  demand-based 
stockage  policies  by  a  more  sophisticated  sparing-to-availability  concept  for 
highly  complex  electronics  systems.  (Section  IV  will  return  to  that  subject.) 

F/A-18A  HORNET  AVIONICS.  Maintainability  and  reliability  were  emphasized 
(including  incentive  awards)  in  the  F-18  program.  The  Weapon  Specification 
included  the  following  BIT  requirements: 

a.  Initiated  BIT.  981  fault  detection;  991  fault  isolation  to  WRA. 

b.  Periodic  BIT.  901  fault  detection;  901  fault  isolation  to  WRA. 

c.  False  Alarm  Rate.  Less  than  11. 

Shortfalls  in  meeting  these  requirements  were  identified  by  Initial 
Operational  Test  and  Evaluation  (October  1980  -  January  1981)  and  Bureau  of 
Inspection  and  Survey  trials  (three  BIT  assessments  from  November  1981  - 
October  1982).  Mature  BIT  assessment  is  currently  in  progress  (October  1983  - 
September  1984).  Due  to  changes  in  the  rules  for  chargeability  of  failures 
and  because  of  major  BIT  engineering  changes  (both  hardware  and  software),  it 
is  difficult  to  get  a  fix  on  actual  BIT  performance  in  terms  of  the  above 
measures.  Based  upon  the  analysis  of  two  data  sources,  the  best  estimate  of 
the  BIT  false  alarm  rate  for  the  AN/APG-65  Radar  is  somewhere  between  45  and 
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62  percent^1.  This  means  that,  on  the  average,  over  50  percent  of  the  radar 
WRAs  pulled  from  the  aircraft  are  not  faulty  or  cannot  be  determined  to  be 
faulty;  i.e.,  NFR  rate  of  50  percent.  This  is  a  poor  result,  given  the 
emphasis  on  maintainability,  but  not  too  different  from  any  other  current 
airborne  radar.  One  lesson  learned  might  be  that  the  specifications  were 
unrealistic  or  even  inconsistent  (high  detection  rates  generally  mean  tight 
tolerances  and,  therefore,  high  false  alarm  rates);  or,  that  further  changes 
in  BIT  software  are  necessary  (e.g.,  false  alarms  can  be  reduced  by  storing  a 
BIT  signal  without  triggering  the  BIT  latch  until  the  BIT  has  been  rechecked 
several  times  confirming  the  failure  indication);  or,  that  avionics  require 
"smart"  BIT.  The  key  lessons  learned,  however,  are  that  testability 
specifications  alone  do  not  solve  the  problem  and  that  the  problem  is  only 
discovered  in  system  integration;  i.e.,  very  late  in  the  full-scale 
engineering  development  phase  when  time  and  funds  are  not  available  to  solve 
unanticipated  problems.  This  explains  the  need  for  a  planned  testability 
maturation  program  in  the  production  phase.  This  is  one  aspect  of  the  F-18 
story. 

Another  aspect  is  the  result  of  the  comprehensive  nature  of  the  avionics 
BIT,  the  integrated  system  architecture  and  ATE  compatibility  design 
requirements.  These  factors  drove  the  various  avionics  subcontractors  (other 
than  the  radar  developer)  to  include  very  effective  BIT  and  testability 
characteristics  in  their  designs.  In  most  cases,  failures  are  identified  at 
the  SRA  level  to  provide  a  WRA  fault-indication  output  to  the  mission 
computer,  maintenance  monitor  panel  and/or  maintenance  signal  data  recorder 
set,  including  identification  of  the  culprit-SRA  involved.  This  (unplanned) 
outcome  has  made  possible  a  change  in  maintenance  concept:  performing 
on-aircraft  fault  isolation  down  to  the  SRA  level  for  many  of  the  nonradar 
avionics  URAs,  using  a  "suitcase"  test  set,  known  as  the  Avionics  Fault  Tree 
Analyzer  (AFTA);  removal  of  the  WRA  and  replacement  of  the  defective  SRA  with 
a  known,  good  SRA;  and  reinstallation  of  the  WRA  on  the  aircraft  followed  by  a 
checkout  via  BIT.  The  concept  eliminates  the  need  for  ATE  to  test  those  WRAs 
and  permits  reduced  stockage  of  spare  WRAs.  About  75  percent  of  all  avionic 
faults  can  be  handled  this  way. 

This  concept  has  been  adopted  by  the  Royal  Australian  Air  Force  and  the 
Canadian  Air  Force  because  of  the  substantial  cost  savings.  The  U.S.  Navy  has 
chosen  not  to  adopt  this  concept.  One  reason  frequently  given  is  that  using 
the  aircraft  as  a  test  bench  is  a  bad  idea,  but  this  would  seem  a  poor 
argument : 


a.  "necessity  to  power-up  the  aircraft"  (counter  argument:  the 
aircraft  is  powered-up  anyway  to  re-run  BIT  for  verification  of  fault 
indication) 


51Data  from  McDonnell  Aircraft  Company  data  bank  (451  BIT  false  alarm 
rate)  and  the  Navy's  Maintenance  and  Material  Management  (3-M)  data  base  (621 
BIT  false  alarm  rate),  May  1983. 
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b.  "unnecessary  operating  hours  imposed  on  installed  equipment  other 
than  the  faulty  WRA"  (counter  argument:  not  true  in  this  case;  the  only 
equipment  which  needs  to  be  turned  on  is  the  indicated  faulty  WRA.  thanks  to 
the  novel  power  control  design  feature  of  the  F-18) 

c.  "impact  on  turnaround  time"  (counter  argument:  if  turnaround  time 
is  critical,  the  WRA  can  be  replaced  and  its  repair  deferred  to  a  shop  or 
reinstalled  and  tested  with  AFTA  when  an  aircraft  becomes  available). 

Another  reason  given  is:  "need  to  reduce  yellow  gear  on  the  flight  deck" 
(AFTA,  though  only  a  small  suitcase  tester,  is  considered  yellow  gear).  But 
the  most  frequently  announced  reason  is  that  this  concept  does  not  conform  to 
the  Navy's  standard  maintenance  concept  for  aeronautical  equipment. 

SUBMARINE  ADVANCED  COMBAT  SYSTEM.  SUBACS  is  in  early  development;  design 
freeze  is  planned  not  earlier  than  FY89.  The  current  maintenance  concept  is 
based  essentially  on  eliminating  at-sea  maintenance,  using  a  mixture  of: 
functional  and  component  redundancy,  automatic  reconfiguration,  and  graceful 
degradation,  sacrificing  the  active  sonar  mode  to  keep  the  passive  sonar  mode 
operational.  In  this  way,  the  plan  is  to  provide  a  70-dav  deployment  with 
zero  maintenance  downtime  for  at  least  the  passive  mode.5'  The  concept  is 
to  limit  shipboard  corrective  maintenance  tasks  to  removal/replacement  of 
faulty  modules  isolated  by  the  built-in  PM/FL  system.  Repair  of  failures  not 
isolated  by  the  system  will  be  deferred  (except  in  emergencies).  The  plan  is 
to  reduce  manning  requirements,  compared  to  those  for  the  SSN  688  Class,  by 
reducing  watch  station  requirements  (more  automation),  reducing  maintenance 
workload  (the  maintenance  concept  moves  most  of  the  maintenance  workload  to 
shore-based  assets),  and  combining  different  ratings  into  one  composite  rating. 

Two  observations  can  be  made  on  the  planned  maintenance  concept.  First, 
the  engineering  effort  required  to  provide  a  fault-tolerant  system  through 
automatic  reconfiguration  of  the  system,  using  standby  equipment,  is  identical 
to  that  required  for  perfect  PM/FL.  If  the  latter  is  technically  possible, 
then  the  use  of  technicians  instead  of  standby  equipment  would  be  much 
cheaper.  If  it  is  not  possible  or  does  not  work,  the  former  (i.e.,  automatic 
reconfiguration)  would  not  work  either.  As  discussed  later  in  this  section, 
fault-tolerant  design  is  not  yet  a  mature  discipline,  but  may  become  so  by  the 
late  1980s  (see  "Future  Technology").  Second,  in  view  of  the  much  higher 
acquisition  cost  associated  with  the  planned  maintenance  concept  (if  it  is  to 
be  successful),  the  approach  taken  appears  to  be  inspired  more  by  the  wish  to 
eliminate  the  need  for  skilled  technicians  than  by  the  objective  of  minimizing 
life-cycle  costs.  The  sarite  crew  reduction  anticipated  with  this  maintenance 
concept  might  be  achieved  without  eliminating  shipboard  technicians  by 
changing  traditional  job  design,  splitting  operator  tasks  from  maintenance 
tasks  and  restructuring  the  ratings. 


52Naval  Sea  Systems  Command,  ILS  Requirements  Document  for  SUBACS.  PMS 
409 ,  February  1982;  Naval  Sea  Systems  Command,  SUBACS  ILS  Requirements. 
Concept  Development  Phase.  PMS  409,  March  1982. 
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If  the  proposed  maintenance  concept  represents  the  way  in  which  the  Fleet 
wants  to  operate,  so  be  it.  The  prime  concern  is  that  the  up-front  costs 
required  for  successful  implementation  of  the  planned  concept  may  be  under¬ 
estimated  or  be  underfunded,  so  that  the  Fleet  ends  up  with  additional 
problems . 

MAINTENANCE  TRENDS.  The  three  examples  summarized  above  may  be  compared  as 
follows : 

a.  The  AEGIS  program  tried  to  resolve  the  maintenance  problem  by 
providing  a  satisfying  solution,  while  focusing  on  the  ultimate  goal  of 
meeting  very  high  A0  requirements.  It  was  successful.  Any  future 
degradation  will  be  due  to  the  personnel  and  training  system  or  inadequate 
supply  support. 

b.  The  F-18  program  tried  to  solve  the  maintenance  problem  by  providing 
an  optimal  solution,  focusing  on  meeting  detailed  testability  requirements. 

It  appears  to  have  failed  in  two  respects.  First,  the  radar  does  not  and 
probably  will  not  meet  the  stated  requirements  because  the  program  was 
unrealistic  with  regard  to  the  resources  needed  to  achieve  those 
requirements.  Second,  the  remaining  avionics  possess  testability  features 
which  permit  a  change  in  maintenance  concept  with  much-reduced  support  costs, 
but  this  potential  is  not  exploited  because  it  conflicts  with  the  traditional 
maintenance  concept. 

c.  The  SUBACS  program  tries  to  dissolve  the  maintenance  problem  by 
eliminating  it  from  the  "0"-level.  The  technology  to  accomplish  that, 
however,  is  immature.  The  costs  to  do  it  successfully  will  be  high,  perhaps 
higher  than  anticipated  and  unaffordable. 

FUTURE  TECHNOLOGY 

INTRODUCTION.  The  examination  of  future  trends  in  technology,  as  applied  in 
combat  systems,  is  focused  on  one  issue:  whether  or  not  future  technology 
will  reduce  maintenance  skill  requirements  compared  to  those  required  today, 
either  by  reducing  maintenance  task  difficulty  (maintenance  complexity,  see 
Section  II)  or  by  reducing  or  eliminating  Fleet  maintenance  requirements. 
Maintenance  skill  requirements  are  defined  in  terms  of  a  composite  of 
cognitive  abilities,  training  duration,  and  hands-on  experience.  Maintenance 
requirements  are  defined  in  terms  of  the  extent  and  amount  of  maintenance 
(maintenance  man-hours  per  operating  hours  or  per  week)  allocated  to  the 
organizational  level  (surface/subsurface)  or  the  organizational  plus 
intermediate  level  (avionics)  by  maintenance  concept. 

This  issue  has  been  examined  within  the  broader  context  of  current  U.S. 
defense  strategy  and  weapon  system  acquisition  philosophy  which,  in 
combination,  result  in  increasing  sophistication  (inherent  complexity,  see 
Section  II)  of  each  next-generation  combat  system  compared  to  its 
<t  predecessor.  Thus,  stated  another  way,  this  analysis  has  examined  the 
prospects  for  maintainability  design  and  its  ability  to  compensate  for 
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increasing  inherent  complexity,  thereby  reducing  maintenance  complexity  below 
that  observed  today. 

The  findings  are  summarized  in  the  following  subsections.  Pointed  out 
are  current  limitations  in  testing  technology  and  factors  which  may  delay 
major  improvements  including:  limitations  of  computer-aided  design  (CAD), 
complexity  of  very  large-scale  integration  (VLSI),  and  cost  of  fault-tolerant 
architecture.  Due  to  uncertainties  about  technological  advances  in  these 
areas,  the  assessment  made  cannot  be  a  forecast,  but  is  couched  in  doubt 
regarding  the  prospects  for  reducing  maintenance  complexity  in  the  near  future. 

CURRENT  STATUS  OF  TESTING  TECHNOLOGY.  Testing  technology  refers  to  the 
technology  associated  with  design  for  testability,  measurement  of  the 
testability  characteristics  of  a  design  from  engineering  drawings,  and 
evaluation  of  the  testability  achieved  by  prototype  hardware.  Testability  is 
loosely  defined  as  the  ease,  extent,  and  accuracy  by  which  system  performance 
can  be  verified  and  failures  detected  and  isolated  to  a  level  of  indenture  in 
consonance  with  the  maintenance  concept.  Lack  of  testability  has  been 
recognized  in  recent  years  as  the  single  key  design  attribute  responsible  for 
maintenance  support  problems.  Weapon  systems  fielded  in  the  1970s,  including 
the  case  examples  examined  in  the  course  of  the  present  study,  were 
characterized  uniformly  (i.e.,  DoD-wide)  by  serious  testability  shortfalls: 
they  were  presumed  to  possess  a  high  degree  of  (semi ) automated  fault  detection 
and  isolation  through  BIT  and  maintenance  diagnostic  software  (typical  values 
in  design  specifications  called  for  90  or  95  percent  fault  isolation),  but  in 
the  operational  environment  these  design  features  did  not  perform  as 
expected.  Maintenance  technicians,  moreover,  were  not  trained  to  cope  with 
this  testability  shortfall;  if  a  task  and  skill  analysis  was  done  at  all,  it 
was  based  on  the  advertised  BIT  performance  so  that  the  training  objectives 
did  not  match  actual  job-task  requirements.  In  some  cases,  testability 
shortfalls  were  reduced  through  engineering  and  software  changes  in  the 
operational  phase  of  a  system's  life  cycle;  in  most  cases  they  were  not. 

Because  of  an  ineffective  feedback  system,  there  was  no  appreciation  of  the 
causes  of  maintenance  performance  shortfalls,  so  that  technical  training 
courses  were  not  revised.  In  other  words,  technicians  were  not  equipped 
(neither  by  training  nor  by  job  aids)  for  their  jobs,  with  the  consequences 
that  could  be  expected. 

This  situation  ultimately  led  to  the  Industry/Joint  Services  Automatic 
Test  Project  initiated  in  1977.  This  was  a  comprehensive,  three-year  study  of 
management  and  technical  issues  related  to  automatic  testing,  both  on-line 
(BIT)  and  off-line  (ATE  and  test  programs).  The  resulting  final  report 
provided  a  program  for  action  in  the  form  of  110  specific  recommendations, 
integrated  into  11  major  areas.  With  respect  to  testability,  rated  the  second 
most  important  area,  the  report  summarized  tae  problem  as  follows: 

"Testability,  as  a  design  discipline,  is  currently  inadequate  because  there  is 
no  accepted  method  for  measuring  it  and  no  mechanism  for  imposing  and 
enforcing  it  during  the  equipment  design  phase."  The  report  recommended 
that:  (1)  verifiable  testability  requirements  be  developed,  (2)  these 
requirements  be  imposed  on  the  prime  system/autoraatic  test  system  design 
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process,  and  (3)  measures  be  taken  to  ensure  compliance. 53  Host  of  the 
recommendations  are  being  followed  up  under  the  aegis  of  the  Joint  Logistic 
Commanders  (JLC)  Panel  on  Automatic  Testing  (established  March  1978)  whose 
study  plan  for  an  Automatic  Testing  Program  was  approved  in  October  1978.  Of 
the  95  subtasks  comprising  that  program,  many  are  directly  or  indirectly 
concerned  with  testability.  The  program  serves  to  coordinate  and  guide  the 
testing  technology  efforts  sponsored  by  the  Services. 

Testability  has  now  emerged  as  a  true  subdiscipline  of  maintainability 
engineering.  The  significant  research  efforts  in  recent  years  have  resulted 
in  a  body  of  knowledge  regarding  the  specification  and  measurement  of 
testability.  Several  guides  have  been  published  (Automatic  Testing 
Acquisition  Planning  Guide,  BIT  Design  Guide,  Selection  Guide  for  Digital  Test 
Program  Generation  Systems,  Reference  Guide  of  ATE  Information  Systems,  Sensor 
Handbooks),  Service  programs  for  future  ATE  have  been  coordinated,  and  a  new 
military  standard  (MIL-STD)  on  testability  is  in  preparation. 54  Training- 
course  materials  have  been  developed  for  industry  (adopted  by  the  National 
Security  Industrial  Association  (NSIA))  and  government  personnel  (incorporated 
in  the  Defense  Systems  Management  College  program  manager  course).  In  spite 
of  these  efforts,  the  impact  on  the  design  engineering  community  so  far 
appears  to  be  limited,  which  is  possibly  caused  by  the  traditional  chasm 
between  design  engineers  and  maintainability  engineers.  The  notion  of 
testability  originated  with  the  automated  testing  community;  i.e.,  logistics 
support  and  maintainability  engineering.  Host  R&D  efforts  to  date  have  been 
limited  to  that  same  community.  To  bring  this  effort  to  fruition,  the 
technology  must  be  transferred,  adopted,  and  institutionalized  by  the  design 
engineering  community  for  application  in  design.  Further,  the  emerging 
technology  requires  much  more  R&D  to  evolve  into  a  mature  engineering 
discipline,  applicable  to  current  and  next  generations  of  electronics 
technology:  VLSI,  very  high-speed  integrated  circuits  (VHSIC),  and 
wafer-scale  integration.  It  appears  that  four  major  thrusts  will  be  necessary 
to  achieve  successful  implementation  of  design  for  testability. 

First,  a  large  educational  effort  will  be  required.  Interviews  with 
prime  defense  contractors  indicate  that  current  experience  in  testability 
design  and  evaluation  is  very  limited. 

Second,  analytical  tools  must  be  developed  to  predict  and  monitor 
testability  in  the  design  phase.  Only  a  handful  of  the  prime  contractors  are 
reported  to  possess  an  effective  capability  (computer-based  models)  to  measure 
testability  at  the  system  level;  most  of  the  software  available  today  actually 
applies  to  modules  (circuit  boards)  or  integrated  circuits  as  part  of  CAD 


1980). 


54MIL-STD-XXX,  "Testability  Program  for  Electronic  Systems  and 
Equipments,"  August  1982  (Draft).  Scheduled  for  approval  in  FY84. 
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systems  (see  "Design  Automation").  The  need  for  a  more  integrated  process, 
both  by  function  (CAD,  system  integration,  failure  mode  and  effects  analysis, 
LSA,  BIT)  and  by  indenture-level  (chip,  board,  assembly,  subsystem,  system), 
has  frequently  been  noted,  but  does  not  currently  exist.  Frequently  mentioned 
models  which  can  be  applied  both  as  a  testability  assessment  tool  in  design 
and  as  a  diagnostic  maintenance  aid  on  the  job,  include  LOGMOD,  STAMP,  and 
FIND.55  But  such  models  are  just  beginning  to  be  considered  as  tools  for 
testability  design.  (Past  emphasis  was  on  their  use  as  maintenance  job 
aids.)  There  is  a  great  deal  of  confusion  about  the  capabilities  or 
limitations  of  these  models.56  Furthermore,  such  models  would  represent 
only  part  of  a  testability  assessment  system;  other  models  are  necessary  for 
generating  the  test  parameters.  While  a  Testability  Analysis  Guide  is  in 
preparation  under  the  auspices  of  the  JLC  Automatic  Testing  Program,  much 
research  remains  to  be  done  with  regard  to  testability  of  embedded  software 
and  hardware-software  integrated  systems.  For  example,  testability  analysis 
has  typically  focused  on  identifying  feedback  loops  and  physically  breaking 
those  loops  through  redesign  or  modifying  test  points  —  an  effort  which  could 
be  easily  verified  in  the  past  through  visual  inspection  of  system  or  module 
block  diagrams.  With  the  increasing  application  of  "embedded  computer 
resources"  in  combat  systems,  an  increasing  portion  of  feedback  loops  are 
time-related  "sneak  paths"  caused  by  software.  Finding  those  "sneak  paths"  is 
impossible  by  traditional  means  and  requires  a  sophisticated  model;  resolving 
those  "sneak  paths"  requires  micro-programming  changes,  not  hardware  changes. 
Sneak  path  analysis  is  currently  more  an  art  than  a  science.  Similarly,  the 
proposed  military  standard  for  testability  is  viewed  with  great  skepticism  by 
testability  engineers  (as  opposed  to  those  who  have  to  administer  testability 


55LOGMOD,  developed  in  the  early  1970s  as  proprietary  software  by  DETEX 
Systems,  Inc.,  is  a  logic  model  algorithm  utilizing  the  maintenance  dependency 
chart  technique,  which  also  underlies  the  format  of  the  more  effective 
troubleshooting  aids.  STAMP  (System  Testability  and  Maintenance  Program), 
developed  by  ARINC  Research  Corporation,  and  FIND  (Fault  Isolation  through 
Nodal  Dependencies),  developed  by  Hughes  Aircraft  Company,  are  more  recent 
models  with  some  similarities  to  LOGMOD.  A  recent  independent  survey  by 
Northrop  Corporation  identified  LOGMOD  as  the  best  testability  assessment  tool 
publicly  available.  A  survey  conducted  by  Lockheed  California  Company  came  to 
the  same  conclusion.  Reportedly,  the  Naval  Ocean  Systems  Center  is  planning 
to  sponsor  a  comparative  evaluation  of  these  three  models. 

56An  evaluation  performed  for  the  JLC  Panel  on  Automatic  Testing 
suggests  that  LOGMOD  is  not  useful  for  analyzing  digital  systems.  See: 

William  L.  Reiner,  "The  Assessment  of  LOGMOD  as  a  Testability  Design  Tool,"  31 
December  1980.  Actually,  most  of  the  LOGMOD  applications  to  date  are  on 
digital  or  hybrid  (sub)systems.  LOGMOD  can,  and  has  been,  applied  to 
electronic  (digital  and  analog)  and  non-electronic  systems,  including  battle 
damage  assessment. 
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programs);  while  it  is  a  necessary  first  step,  they  do  not  see  it  as  an 
effective  approach  to  solve  the  testability  problem. 5? 

Third,  the  emphasis  in  weapon  system  acquisition  must  shift  from 
design-to-unit-production-cost  to  life-cycle  cost.  While  the  latter  is  formal 
policy,  it  is  accorded  little  more  than  lip  service.  Design  for  testability 
will  increase  the  front-end  cost  as  a  result  of  more  thorough  engineering 
analysis  requirements  and  additional  cost  for  BIT  hardware  and  software.  It 
may  also  reduce  system  reliability  due  to  increased  complexity  caused  by  more 
BIT  circuitry  (compared  to  poorly  designed  BIT,  however,  there  may  be  no 
reduction  in  reliability). 

Fourth,  a  new  acquisition  concept  is  necessary  to  guide  implementation  of 
design  for  testability,  with  the  associated  changes  in  current  acquisition 
policies.  This  new  concept  is  known  as  "integrated  diagnostics,"  a  concept 
adopted  by  the  Air  Staff  in  1981  for  implementation  by  the  Air  Force  Systems 
Command  and  currently  being  examined  for  DoD-wide  implementation  by  various 
JLC  and  NSIA  committees . 58  In  essence,  the  concept  is  that  100  percent 
diagnostics  is  required;  must  be  quantitatively  allocated  to  BIT, 
proceduralized  aids,  and  the  maintainer's  cognition  (training)  based  on  early 
trade-off  studies;  must  be  measured  and  monitored  throughout  the  development 
cycle  for  all  levels  of  maintenance;  must  be  matured  through  a  planned 
maturation  program  in  the  production  phase;  and  must  be  validated  and  verified 
in  the  production  phase.  This  concept  has  not  yet  been  adopted  and 
institutionalized  through  DoD  and  Service  regulations. 

In  sum,  the  current  status  of  testing  technology  is  immature.  Much 
remains  to  be  done  to  bring  it  to  fruition.  The  OSD-IDA  R&M  Study  (see 
Section  I,  footnote  3)  arrived  at  a  similar  conclusion.  The  report  describes 
the  current  status  of  testing  technology  and  identifies  R&D  needs  in  the 
various  testing  technology  areas.  The  study  recommendations  are  quoted  as 
follows : 

a.  Initiate  a  major  weapon  system  design  technology  program,  which 
injects  testing  technology  into  the  design  process. 

b.  Invest  in  expanding  the  testing  technology  base  to  provide 
"off-the-shelf"  proven  alternatives  for  use  in  weapon  system  design. 

c.  Institutionalize  the  transitioning  and  utilization  of  testing 
technology. 

d.  Initiate  actions  to  improve  testing  technology  management. 


5?John  W.  Cunningham,  BITE,  Inc.,  personal  communication,  1983. 
^Proceedings  of  Conference  on  Integrated  Diagnostics,  February  1983. 
^^ida  Document  No.  D-41,  Testing  Technology.  April  1983  (Draft). 
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Another  indication  of  the  immature  status  of  testing  technology  is 
provided  by  the  current  R&D  program  of  the  Rome  Air  Development  Center  (one  of 
the  major  sponsors  of  testability  research),  which  includes  the  following  four 
projects : 

a.  Prediction  and  analysis  of  testability  attributes. 

b.  Development  of  a  practical  engineering,  mathematically  tractable 
verification  technique  for  accurate  and  economical  test  and  demonstration  of 
BIT. 


c.  Development  of  practical  design  concepts  to  increase  the  efficiency, 
productivity,  and  cost-effectiveness  of  BIT  through  the  application  of 
artificial  intelligence  (AI)  techniques  ("Smart  BIT"). 

d.  Development  of  design  tools  to  implement  the  Integrated  Diagnostics 
policy  in  an  efficient  and  cost-effective  manner. 

DESIGN  AUTONATION.  CAD  has  evolved  as  a  design  tool  to  assist  the  design 
engineer  with  many  routine  tasks.  In  the  late  1950s,  the  focus  was  on 
automated  logic  diagrams,  back-panel  wiring  with  discrete  wires,  and 
electrical  load  checking.  With  the  introduction  of  PC  boards  and  increasing 
density  of  discrete  components  in  the  1960s,  the  focus  shifted  toward  circuit 
analysis  and  simulation  techniques  for  discrete  circuits.  Integrated  circuits 
( ICs )  arrived  by  the  early  1970s,  so  that  computers  became  indispensable  as 
record-keeping  tools  and  means  of  design  verification.  By  the  mid-1970s, 
design  tools  evolved  to  automatically  generate  physical  layout  and 
interconnections  of  the  components  within  an  IC. 

The  present  situation  is  characterized  by  numerous  loosely  coupled  CAD 
programs,  mostly  incompatible  due  to  different  data  formats,  requiring  manual 
intervention  to  move  from  one  program  to  another.  Much  of  the  design  process 
is  still  manual,  with  the  designer  creating  specific  circuit  designs  at  the 
logic  gate  level,  but  using  CAD  tools  for  automated  testing  and  simulation  of 
circuitry,  verification  of  physical  and  functional  design,  and  production  of 
final  layouts  for  IC  2nd  complex  PC  board  fabrication.  With  the  introduction 
of  VLSI,  it  has  become  apparent  that  the  quagmire  of  stand-alone  CAD  tools 
must  be  replaced  by  an  integrated  design  automation  system.  The  consensus  is 
that  current  tools  are  no  longer  adequate  and  are  the  constraining  factor 
limiting  the  growth  of  IC  complexity.  Integrated  design  automation  has  become 
a  "critical  technology"  to  meet  the  needs  of  VLSI  in  the  1980s. 60 

The  status  and  problems  in  design  automation,  especially  in  the  area  of 
design  for  testability,  are  best  summarized  by  industry  spokesmen: 


60Lawrence  M.  Rosenberg,  "The  Evolution  of  Design  Automation  to  Meet 
the  Challenges  of  VLSI,"  17th  Design  Automation  Conference  Proceedings,  June 
1980. 
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a.  Rosenberg,  (RCA):  Our  studies  have  shown  that  design  for  testability 
using  currently  understood  techniques  can  be  quite  expensive;  more  so  than 
their  proponents  acknowledge.  In  addition,  conventional  testability  measures, 
such  as  "stuck-at"  fault-coverage  monitors  {used  by  all  known  parallel  and 
concurrent  fault  simulation  programs)  do  not  address  the  issue  of  pattern- 
sensitive  faults.  Nor  do  the  scan  techniques  test  for  them.  ...  I  believe 
the  development  of  cost-effective  design  for  testability  approaches  and  their 
supporting  Design  Automation  tools  remain  a  large  challenge. 63 

b.  Fitch  (IBM):  As  system  and  component  tehnologies  continue  their 
rapid  rate  of  development,  there  is  an  increasingly  higher  probability  that 
our  ability  to  capitalize  on  them  will  be  severely  impacted  by  the  lack  of 
appropriate  design  system  technology  development.  .  .  .  The  great  progress 
made  in  the  field  (of  Design  Automation)  over  the  past  10  years  is  miniscule 
compared  to  what  we  must  accomplish  in  the  next  five  years. 6^ 

c.  Gwyn  (Sandia  National  Laboratories):  Substantial  progress  was  made 
in  implementing  design  aids  during  the  1970s;  however,  the  development  of  new 
aids  must  be  accelerated  during  the  1980s  to  support  circuit  design  of  VLSI 
complexity.  New  design  aids  will  be  required  because  of:  (1)  increased 
design  complexity,  (2)  scarcity  of  design  engineers,  (3)  need  to  reduce 
circuit  design  time  and  cost,  and  (4)  increased  emphasis  in  producing 
error-free  manufacturable  designs.  .  .  These  design  and  manufacturing  problems 
can  be  solved  only  by  increasing  the  use  of  Design  Automation  and  improving 
the  design  aids.  ...  In  the  future,  new  aids  will  evolve  and  the  emphasis 
will  be  on  integrated  design  systems.63 

d.  Sapiro  (American  Microsystems ,  Inc.):  VLSI  design  demands  a 
hierarchical  design  approach  with  tools  for  each  level  of  abstraction  in  the 
hierarchy.  ...  We  see  the  1980s  as  broken  up  into  two  segments  —  the  early 
1980s,  when  the  work  started  on  physical  design  tools  in  the  late  '70s  will 
bear  fruit,  and  the  late  '80s  when  a  shift  will  occur  away  from  physical 
design  tools  toward  front-end  tools  to  help  the  system  design  cope  with 
greatly  increased  density  and  complexity.  All  these  tools  must  be  glued 
together  by  the  much  touted  hierarchical  design  style.  .  .  .  These  [front  end 
design]  tools,  which  will  have  the  greatest  impact  on  the  system/circuit 
design  process,  are  the  most  complex  to  develop.  We  see  concentrated  efforts 
starting  in  the  1980s  but  not  completed  until  the  1990s.64 


61Ibid. 

6?Posi  tion  paper  included  in  the  17th  Design  Automation  Conference 
Proceedings ,  June  1980. 

63Ibid. 

64Ibid. 
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e.  Goel  (IBM):  We  define  Very  Large  Scale  Logic  (VLSL)  to  be  that 
logic  containing  more  than  10,000  gates.  VLSL  may  not  be  common  on  the 
semiconductor  chip  of  today  but  is  found  in  abundance  on  second  or  higher 
level  packages.  Thorough  testing  of  chips  prior  to  packaging,  combined  with 
the  availability  of  probe  points  on  higher  levels  of  packaging,  have  combined 
to  temporarily  obscure  the  testing  problem  for  cards  or  boards  with  more  than 
10,000  gates.  However,  when  VLSL  becomes  commonplace  on  chips,  the  chip 
testing  problem  will  certainly  become  severe  since  a  chip  cannot  be  probed. 
Sizable  increases  in  chip  power  dissipation  may  necessitate  cooling  methods 
which  all  but  eliminate  the  feasibility  of  providing  probe  points  on  higher 
level  packages.  .  .  .  Automatic  test  generation  for  VLSL  will  continue  to  rely 
on  design  for  testability  structures.  The  Level  Sensitive  Scan  Design  (LSSD) 
structure  is  an  outstanding  example  of  a  design  structure  for  testability.  .  . 
[But]  with  test  generation  costs  of  the  order  of  minutes  of  CPU  time  for  a 
1,000  gate  structure,  we  project  a  CPU  time  upwards  of  1,000  hours  for  the 
100,000  gate  structure  —  assuming  super  test  pattern  generation  techniques 
and  the  deductive  fault  simulator.  Parallel  fault  simulation  is  a  technique 
which  must  be  abandoned  for  large  logic  structures,  considering  that  a  1 
million  fold  increase  in  costs  will  be  incurred  in  going  from  a  1,000  gate  to 
the  100,000  gate  logic  structure.  We  hope  that  this  paper  will  cause  some 
rethinking  about  the  frequently  heard  statement  that  test  generation  for 
combinational  logic  circuits  is  a  solved  problem. ^5 

An  independent  review  of  the  status  of  design  automation  in  18  companies 
and  government  laboratories,  conducted  in  1979,  came  to  the  following 
conclusions . 66 

a.  Testing  is  one  of  the  less  mature  but  most  active  areas  of  Design 
Automation  development. 

b.  Little  effort  is  being  devoted  to  formal  design  verification; 
rather,  designs  are  checked  in  an  ad  hoc  fashion  by  simulation,  by  building 
prototypes,  or  by  employing  the  "first  built"  machine. 

c.  Host  companies  are  either  not  ready  to  deal  with  VLSI  design 
problems  or  are  reluctant  to  discuss  their  present  and  planned  efforts  in 
VLSI.  Register  Transfer  Level  simulation,  design  verification,  layout,  design 
for  testability,  and  test  generation  must  be  strenghthened  if  future  Design 
Automation  Systems  are  to  take  advantage  of  new  technologies. 


65Prabhakar  Goel,  "Test  Generation  Costs  Analysis  and  Projections," 

17th  Design  Automation  Conference  Proceedings.  June  1980. 

66Helvin  A.  Breuer,  et  al.,  "A  Survey  of  the  State  of  the  Art  of  Design 
Automation,"  Computer.  Vol.  14,  No.  10  (October  1981),  pp.  58-75. 
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Several  universities  (especially,  Carnegy-Mellon  University)  and 
government  laboratories  (especially,  Lawrence  Livermore  Laboratory)  are  active 
in  the  field  of  Design  Automation.  The  Air  Force  Institute  of  Technology 
recently  embarked  on  a  five-year  RAD  program  to  upgrade  its  CAD  tools  into  an 
integrated  design  automation  system.6' 

In  sum,  the  requirement  to  design  and  produce  testable  electronic 
components  is  widely  recognized,  but  current  tools  are  not  adequate  to  cope 
with  VLSI  complexity.  According  to  the  judgment  of  the  above  experts,  it 
would  be  1990  at  the  earliest  before  such  tools  become  available.  In  the 
meantime,  the  VHSIC  program,  as  described  next,  has  accelerated  RAD  on  CAD 
tools  and  integrated  design  automation  systems.  Based  on  the  progress 
reported  to  date  and  planned  RAD  efforts  through  FY87,  it  would  appear  that  by 
the  late  1980s  design  automation  will  have  matured  enough  to  cope  with  VLSI 
complexity. 

VHSIC  PROGRAM.68  The  VHSIC  program  is  DoD's  highest  priority  technology  RAD 
program.  The  program  was  originally  formulated  by  DoD  in  1978  as  a  strategy 
to  counter  the  lag  of  electronics  applications  in  defense  equipment  behind 
commercial  micro-electronics  technology  experienced  throughout  the  1970s.  The 
reasons  given  for  the  program  were  to:  (1)  reverse  the  erosion  of  the  U.S. 
electronic  technology  lead  in  the  world,  (2)  increase  the  exploitation  of  IC 
technology  advances,  and  (3)  focus  development  efforts  on  military 
requirements.  The  VHSIC  program  differs  from  commercial  VLSI  developments  by 
putting  emphasis  on: 

a.  development  of  ICs  for  broad  classes  of  military  systems,  including 
functions  not  needed  in  commercial  applications, 

b.  increased  real-time  system  throughput,  requiring  higher  chip 
complexity  and  higher  clock  rates, 

c.  architectural  concepts  minimizing  the  need  for  design  customization, 

d.  reliability  requirements  in  the  military  environment  (temperature 
range,  radiation  exposure), 

e.  exploitation  of  higher  chip  complexity  for  chip  self-test  (BIT)  and 
fault-tolerant  design. 

The  program  structure  and  goals  are  summarized  in  Table  5.  Recent 
revisions  in  the  program  were  designed  to  emphasize  VHSIC  Phase  1  insertions 


,1 


67Harold  W.  Carter,  "A  Plan  for  Digital  Systems  Design  Automation  at 
the  Air  Force  Institute  of  Technology,"  Planning  Document  (Dayton,  Ohio:  Air 
Force  Institute  of  Technology,  Wr ight-Patterscn  Air  Force  Base,  November  1981). 

68The  VHSIC  program  description  is  taken  from  Larry  W.  Sumney,  "VHSIC  - 
A  Status  Report,"  IEEE  Spectrum,  pp.  34-39,  December  1982. 
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way  the  contracts  are  written.  While  the  Navy  tailors  its  contract 
specifications  to  the  developmental  system  using  type  specifications  which 
involve  standard  general  specifications,  the  extent  of  specificity  and  detail 
of  human  resource  constraints  in  contract  specifications  ultimately  is 
determined  by  policy  and  approved  military  standards  and  specifications.  As 
indicated  above  none  of  the  design  guides  so  far  has  been  accepted  as  a 
military  standard;  the  only  one  available,  MIL-STD-1472,  is  of  little  use 
beyond  the  traditional  anthropometric  measures  of  human  factors.  And  current 
Navy  policy  leaves  too  much  leeway  to  the  program  (project)  manager.87  To 
influence  design  in  a  more  substantial  way,  design  guides  must  be  adopted  as 
military  standards  which  are  invokable  in  contracts.  And  before  this  can 
happen,  a  consensus  must  evolve  about  what  data  in  what  format  are  necessary 
to  relate  ski 11  and  skill  level  to  design  attributes.  Such  a  consensus, 
apparently,  does  not  exist  at  the  present  time. 

TESTABILITY.  The  above  reasons  explain  why  design  for  maintainability  has  not 
been  very  successful  in  the  past,  and  may  offer  little  prospect  for 
improvements  in  the  future  unless  the  entire  approach  to  influencing  equipment 
design  is  changed.  The  notion  among  human  factors  specialists  is  that  skill 
levels  and  skills  can  be  defined,  quantified,  and  presented  to  the  design 
engineer  as  contractual  constraints;  and  that  just  some  more  R&D  is  necessary 
to  make  that  approach  possible.  Instead,  a  more  pragmatic  approach  would  be 
to  focus  on  the  one  equipment  design  characteristic  which  determines 
maintenance  task  complexity  that  planned  maintenance  personnel  can  handle  (a 
function  of  experience,  training,  and  basic  abilities)  based  on  empirical 
evidence.  That  one  equipment  characteristic  is  testability,  i.e.,  the  ease, 
extent,  and  accuracy  by  which  system  performance  can  be  verified  and  failures 
detected  and  isolated  to  a  level  of  indenture  in  consonance  with  the 
maintenance  concept.  This  is  the  approach  adopted  by  the  1983  revision  of 
HIL-STD-470. 

The  current  status  and  prospects  of  testing  technology  (design  for 
testability)  is  reviewed  in  Section  III.  Design  for  testability  is  crucial 
for  achieving  any  substantial  improvements  in  maintenance;  in  its  absence, 
training  and  aiding  improvements  will  be  very  costly  and  produce  only  marginal 
returns  if  measured  in  increased  A0.  The  Navy  is  on  the  record  as 
recognizing  that  testability  must  be  accorded  top  priority  and  funded 


87NAVMAT  Instruction  3900. 9A,  "Human  Factors  in  System  Development," 

1981  (Draft).  This  instruction  exhibits  a  lack  of  detailed  HFE  standards, 
criteria,  specifications,  and  guidance.  A  checklist  of  human  factors 
information  requirements  in  the  acquisition  process  is  provided  by: 

D.  Keister,  Behavioral  Inputs  to  the  Weapon  System  Acquisition  Process.  NPRDC 
SR  83-21  (San  Diego:  Navy  Personnel  Research  and  Development  Center,  March 
1983) . 
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underutilization  of  man-machine  technology  in  system  acquisition.  A  similar 
assessment  was  made  by  the  General  Accounting  Office  (GAO)  which  estimated 
that  over  50  percent  of  weapon  system  failures  are  human-induced  (operator  or 
maintainer)  and  concluded  that  more  attention  should  be  paid  to  human  factors 
during  the  design  process.  It  noted,  however,  that  the  new  policy  emphasis  on 
human  limitations  in  the  design  of  systems  may  have  a  very  limited  impact 
because: 

a.  Human  factor  specifications,  standards,  and  handbooks  used  in 
designing  and  developing  systems  and  equipment  do  not  adequately  address  human 
limitations. 

b.  There  are  no  common  methodologies  and  data  sources  for  use  by  system 
designers  in  forecasting  skill  levels  of  future  military  personnel. 

c.  DoD  testing  policies  and  procedures  do  not  tend  to  identify  and 
resolve  potential  human-induced  failures  during  the  developmental  stages  of 
the  acquisition  process.86 

NAVSEA  has  sponsored  a  HFE  technology  program  since  1975,  but 
implementation  of  the  recommended  process  in  new  acquisition  programs  depends 
on  available  program  funding.86  The  most  straightforward  design  guide  in 
the  Navy  relating  design  attributes  to  skill  levels  is  the  one  previously 
referred  to  (see  footnote  83). 

DESIGN  CONTRACTS.  With  the  requisite  acquisition  policies  in  place,  and 
information  available  to  guide  the  design  effort  toward  a  system  maintainable 
by  Service  technicians,  the  outcome  may  still  be  a  system  that  is  difficult  to 
maintain.  The  reason  for  this  is  the  design  contract.  Unless  human  resource 
constraints  are  explicitly  included  as  design  requirements  (not  just  goals), 
there  is  little  incentive  for  the  design  group  to  pay  more  than  lip  service  to 
this  aspect  of  design.  It  is  difficult  enough  to  meet  the  cost,  performance, 
and  schedule  requirements  which  are  invariably  "hard"  specifications.  While 
the  system  program  manager  can  influence  the  design  effort  through  the 
preliminary  and  critical  design  review,  any  leverage  ultimately  depends  on  the 


86U.S.  General  Accounting  Office,  Effectiveness  of  U.S.  Forces  Can  Be 
Increased  Through  Improved  Weapon  System  Design,  PSAD-81-17  (Washington,  D.C.: 
Comptroller  General  of  the  United  States,  January  1981).  It  reiterated  the 
same  statement  in:  Guidelines  for  Assessing  Whether  Human  Factors  Were 
Considered  in  the  Weapon  Systems  Acquisition  Process,  FPCD-82-5  (Washington, 
D.C.:  U.S.  General  Accounting  Office,  December  1981). 

86T.  B.  Malone,  et  al.,  Human  Factors  Engineering  Technology  for  Ship 
Acquisition  (Naval  Sea  Systems  Command,  October  1976).  This  was  expanded 
subsequently  to  include  aviation  material:  C.  J.  Baker,  et  al.,  Human  Factors 
Engineering  for  Navy  Major  Weapon  System  Acquisition  (Naval  Air  Development 
Center  and  Naval  Sea  Systems  Command,  July  1979). 
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System  (NEPDIS),  currently  under  development.  A  recent  review  of  the 
literature  on  the  use  of  human  resource  data  to  influence  equipment  design 
concluded  that  a  solution  to  the  problem  is  not  clearly  definable  and  the 
types  of  data  required  for  communicating  human  resource  information  to  design 
engineers  not  readily  identifiable. 81 

The  format  normally  used  for  presenting  design-relevant  human  resource  or 
human  factors  data  to  the  design  engineer  is  a  design  guide.  Since 
publication  of  the  first  comprehensive  guide  (see  footnote  74),  literally 
hundreds  have  been  published.  None  have  been  adopted  as  a  military  standard. 
The  only  existing  standard  is  primarily  concerned  with  anthropometric  data, 
not  with  such  concepts  as  skill  and  skill  level  and  their  relationship  to 
design  features.82  A  recent  design  guide  for  surface  ship  equipment, 
however,  is  focused  especially  on  relationships  between  design  attributes  and 
skill  levels  (pay  grades)  and  is  probably  as  specific  as  one  could  expect, 
given  the  current  state  of  knowledge.83 

HUMAN  FACTORS  ENGINEERING.  Human  factors  engineering  (HFE)  is  broadly  defined 
as  the  discipline  concerned  with  the  design  and  evaluation  of  the  "man-machine 
interface"  to  ensure  that  it  accommodates  people's  capabilities  and 
limitations.  HFE,  as  applied  in  the  system  acquisition  process,  is,  by 
specification,  supposed  to  encompass  all  elements  of  that  interface:  human 
engineering,  biomedical  aspects,  personnel  requirements,  task  and  workload 
analysis,  training,  technical  manuals  and  other  performance  aids,  and 
test/evaluation.84  While  that  specification  has  all  the  right  words,  there 
is  little  guidance  how  the  human  factors  program  requirements  are  to  be 
accomplished.  The  only  data  reference  in  this  specification  is  HIL-STD-1472, 
and  the  weight  of  that  standard  is  on  operability  issues  (design  of  displays, 
workspace,  environment,  safety),  not  maintainability  (15  pages  out  of  200-page 
standard  are  concerned  with  maintainability,  including  two  pages  on  test 
points,  test  equipment,  and  failure  indicators).  As  a  result,  HFE  has  not 
been  very  effective  in  improving  design  for  maintainability.  The  NRAC  Study 
Group,  previously  referred  to  (see  Section  I),  concluded  that  the  existing 
military  specification  and  standard  are  inadequate,  thus  contributing  to  the 


83Dennis  Sullivan,  System  Design  Characteristics  and  User  Skills:  A 
Literature  Review.  NPRDC  TN  81-9  (San  Diego:  Navy  Personnel  Research  and 
Development  Center,  March  1981). 

82MIL-STD-1472,  "Human  Engineering  Design  Criteria  for  Military 
Systems,  Equipment  and  Facilities,"  February  1968  (latest  revision,  May  1981). 

83R.  A.  Dick  and  Marvin  C.  McCallum,  Anacapa  Sciences,  Inc.,  Design 
Guiae  to  Operator  and  Technician  Requirements.  NPRDC  SR  83-14,  6  vols.  (San 
Diego:  Navy  Personnel  Research  and  Development  Center,  February  1983). 

84HIL-H-46855 ,  "Human  Engineering  Requirements  for  Military  Systems, 
Equipment  and  Facilities,"  March  1968  (latest  revision,  January  1979). 
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supportability  to  the  coequal  of  cost,  performance,  and  schedule.7®  The 
second  reason  was  that  LSA  typically  was  not  conducted  until  the  full-scale 
engineering  development  phase;  i.e.,  much  too  late  to  influence  design.  The 
applicable  specification  was  vague  about  the  conduct  of  LSA  in  earlier  phases 
of  the  acquisition  cycle.  Thus,  LSA  had  become,  in  practice,  a  make-work, 
activity  of  filling  out  reams  of  paper,  instead  of  the  design-analytic  tool  as 
had  been  intended.'9  Revisions  of  MIL-STD-1388  were  initiated  in  1981, 
emphasizing  the  need  for  front-end  analysis  and  iteration  of  LSA  from  broad 
system-level  analysis  to  detailed  component-level  analysis  throughout  the 
acquisition  cycle.  The  revised  standard  was  only  recently  approved  and 
promulgated  (April  1983).  The  third  reason  was  that  past  policy  focused  on 
elapsed  maintenance  time  (e.g.,  mean-time-to-repair,  or  some  percentile  of 
repair-time  distribution)  as  the  sole  measure  of  maintainability.  Outside  the 
defense  acquisition  community,  there  is  a  concensus  that  this,  by  itself,  is  a 
poor  criterion  measure  of  maintainability:  it  is  insensitive  to  design 
variables,  downtime  is  contaminated  with  factors  over  which  the  maintainer  has 
little  control,  etc.80  There  is,  however,  no  consensus  on  what  measures 
should  be  used  instead.  The  recent  revision  of  MIL-STD-470  ("Maintainability 
Program  for  Systems  and  Equipment,"  January  1983)  adds  testability  as  a 
measure  of  maintainability.  This  fundamental  change  may  alleviate  much  of  the 
problem. 

HUMAN  RESOURCE  DATA.  The  lack,  of  specificity  of  human  resource  data  is 
another  reason  why  design  for  maintainability  is  hard  to  achieve.  Definitions 
of  skills  and  skill  levels  are  vague  and  inconsistent;  available  data  are 
unclear  and  difficult  to  translate  into  engineering  specifications;  and  there 
is  a  lack  of  concrete  personnel  skill  data  applicable  to  the  design  of  Navy 
systems.  The  Navy  Occupational  Task  Analysis  Program  data  base  appears 
inadequate  for  this  purpose;  this  also  appears  to  be  the  case  for  the  more 
detailed  data  base,  the  Navy  Enlisted  Professional  Development  Information 


78DoD  Directive  5000.1,  "Major  System  Acquisitions,"  March  1980  (and 
subsequent  revisions). 

DoD  Instruction  5000.2,  "Major  System  Acquisition  Procedures,"  March 
1980  (and  subsequent  revisions). 

DoD  Directive  5000.39,  "Acquisition  and  Management  of  Integrated 
Logistic  Support  for  Systems  and  Equipment,"  January  1980  (superseding  DoD 
Directive  4100 . 35 ) . 

DoD  Directive  5000.40,  "Reliability  and  Maintainability,"  July  1980. 

79"The  LSA  shall  be  the  single  analytical  logistic  effort  within  the 
systems  engineering  process,  and  shall  be  responsive  to  acquisition  program 
schedules  and  milestones."  MIL-STD-1388,  "Logistic  Support  Analysis,"  October 
1973. 


80Jul  ien  M.  Christensen,  "Summary  Comments,"  Proceedings  of  Human 
Factors  Design  for  Maintainers  Conference  (Warminster,  Pennsylvania:  Naval 
Air  Development  Center,  March  1982). 
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in  1963. 74  By  the  early  1960s,  maintainability  engineering  had  evolved  as  a  . 
true  engineering  discipline:  maintainability  was  defined  in  quantitative 
terms;  methods  for  prediction  and  measurement  were  developed;  and,  for  the 
first  time,  it  began  to  be  specified  as  a  design  requirement  in  the 
acquisition  process.75  The  first  systems-oriented  textbook,  on 
maintainability,  relating  it  to  system  effectiveness  (A0)  and  ownership 
cost,  was  published  in  1964. 76  And  maintainability  was  recognized  as  the 
counterpart  of  reliability  with  both  identified  as  key  elements  in  the  new 
concept  of  integrated  logistics  support.77 

Thus,  in  the  early  1960s,  most  of  the  machinery  (policy,  guidance,  and 
knowledge)  was  in  place  to  accommodate,  if  not  encourage,  design  criteria  for 
equipment  maintainability  considering  the  intended  users.  Yet,  equipment 
design  characteristics  have  remained  one  of  the  chief  causes  of  maintenance 
support  problems.  The  reasons  for  this  lack  of  success  in  fielding  better- 
maintainable  equipment  may  be  classified  in  four  categories:  (1)  the 
acquisition  process,  (2)  the  inexactness  of  human  resource  data  such  as  skill 
and  skill  level,  (3)  the  focus  of  human  factors  engineering  an  operability 
vice  maintainability,  and  (4)  the  way  human  resource  constraints  are  specified 
in  design  contracts.  Recent  fundamental  changes  in  acquisition  policies, 
however,  offer  the  potential  of  major  improvements  in  design  for 
maintainability. 

ACQUISITION  PROCESS.  Three  basic  reasons  explain  why  design  engineers 
generally  paid  little  attention  to  human  design  criteria  for  maintainability 
until  DoD  changed  its  acquisition  policies  in  the  late  1970s.  One  reason  was 
that  program  cost,  performance,  and  schedule  were  accorded  top  priority  by 
formal  policy,  with  maintainability  or  supportability  a  distant  fourth.  This 
was  corrected  with  the  1980  revisions  in  acquisition  policy,  elevating 


74C.  T.  Morgan,  et  al.,  (eds.),  Human  Engineering  Guide  to  Equipment 
Design  (New  York:  McGraw-Hill,  1963).  Revision  of  this  guide  began  in  1965 
and  was  completed  in  1972  under  the  editorship  of  Harold  P.  Van  Cott  and 
Robert  G.  Kinkade,  published  by  the  U.S.  Government  Printing  Office.  Attempts 
to  adopt  this  revised  guide,  one  of  the  most  referenced  documents  for  this 
subject,  as  a  military  standard,  have  been  unsuccessful. 

75MIL-STD-778,  "Maintainability  Terms  and  Definitions,"  April  1964. 
MIL-STD-470,  "Maintainability  Program  Requirements,"  March  1966. 
MIL-STD-471 ,  "Maintainability  Demonstration,"  February  1966. 
MIL-HDBK-472 ,  "Maintainability  Prediction,"  May  1966. 

76A.  S.  Goldman  and  T.  B.  Slattery,  Maintainability:  A  Major  Element 
of  System  Effectiveness  (Huntington,  New  York:  Robert  E.  Kruger  Publishing 
Co.,  1964). 

77DoD  Directive  4100.35,  "Integrated  Logistics  Management  System,"  June 

1964. 
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l.  Job  Aids.  The  potential  role  of  paper  job  aids  in  the  performance 
of  combat  system  maintenance  is  commonly  overestimated:  the  operational 
environment  at  the  organizational  maintenance  level  (aboard  deployed  units) 
limits  the  acceptance  and  utilization  of  such  aids  because  they  tend  to  slow 
the  work.  The  chief  benefit  of  electronic  maintenance  aids  lies  in  their 
potential  flexibility  so  that  the  drawbacks  of  paper  aids  can  be  avoided. 
Electronic  aids  can  be  made  to  fit  the  job-site  requirements  (size, 
portability)  and  match  the  user's  needs  (directive  task  instructions  or 
tutorial  information).  The  Navy  has  been  slow  in  recognizing  the  potential  of 
such  aids  and  in  adopting  them  for  operational  use. 

m.  Training  Equipment.  Compared  to  operator  trainers  (especially 
flight  simulators),  experience  with  maintenance  trainers  is  somewhat  limited. 
There  is  a  lack  of  institutional  knowledge,  procedures,  and  guidelines  on  how 
to  develop  design  specifications  for  maintenance  trainers  such  that  positive 
transfer-of-training  is  guaranteed  at  least  cost.  From  the  limited 
experimental  data  available,  it  is  known  that  maintenance  trainers  can  be  very 
cost-effective. Most  technical  training  provided  today  uses  operational 
equipment,  not  maintenance  training  equipment.  The  result  is  ineffective 
hands-on  training  and  excessive  costs. 

A  brief  discussion  of  these  issues  follows  to  provide  the  basis  for  the 
recommendations  in  Section  V. 

EQUIPMENT  DESIGN 

How  to  influence  the  design  of  new  military  equipment  in  order  to  ensure 
it  will  be  supportable  by  the  human  resources  that  are  available  is  an  age-old 
issue.  A  brief  review  of  what  has  evolved  in  this  area  suggests  that  little 
progress  has  been  made  over  the  past  20  years,  at  least  until  very  recently. 
Improvements  in  the  acquisition  process,  procedures,  and  guidelines  for 
maintainability  have  been  substantial  indeed,  but  largely  inadequate  to  keep 
abreast  of  "technology  push"  (increasing  technological  complexity  of  the 
equipment)  and  its  impact  on  maintenance  skill  requirements. 

Most  of  the  groundwork  in  maintainability  and  human  engineering  was  laid 
in  the  1950s.  In  1954,  a  Joint  Services  Steering  Committee  was  convened  to 
sponsor  development  of  a  ",  .  .  guide  in  human  engineering  which  the  designer 
can  use  in  the  same  manner  as  handbooks  in  other  areas  to  assist  in  solving 
design  problems  as  they  arise.  ..."  This  guide  was  completed  and  published 


73J.  Orlansky  and  J.  String,  Cost-Effectiveness  of  Maintenance 
Simulators  for  Military  Training,  IDA  Paper  P-1568,  Institute  for  Defense 
Analysis,  Arlington,  VA,  August  1981. 
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requirement  of  combat  system  maintenance  technicians.  This  is  not  an 
indictment  of  ISD,  as  such,  but  the  way  it  has  been  implemented  by  the  Navy. 

g.  Proficiency  Monitoring.  Two  steps  are  key  in  any  instructional 
system,  whether  or  not  based  on  the  ISD  model.  Those  are  task  analysis  (to 
identify  training  needs)  and  job  performance  monitoring  (to  provide  feedback 
to  the  instructional  system).  Neither  is  done  well  in  the  Navy.  Task 
analysis,  if  done  in  accordance  with  current  ISD  procedures,  is  limited  to 
tasks  which  can  be  "proceduralized, "  ignoring  cognitive  tasks.  Job 
performance  may  be  monitored  locally  on  a  subjective  basis,  but  the  Navy  lacks 
a  standardized  methodology.  Lack  of  feedback  inhibits  identification  of  skill 
deficiencies  which  could  be  corrected  through  refresher  training,  remedial 
training,  or  revised  training  curricula.  Questionnaires  currently  in  use  do 
not  provide  effective  feedback. 

h.  Personnel  Classification.  Rating  assignment  eligibility  is 
determined  solely  on  the  basis  of  ASVAB  test  scores,  yet  the  ASVAB  measures 
only  a  few  of  the  individual  abilities  related  to  job  performance  potential. 
The  absence  of  adequate  tests  inhibits  both  optimal  utilization  of  the 
available  talent  pool  and  implementation  of  adaptive  training  to  improve 
training  effectiveness  (reduce  attrition). 

i.  Maintenance  Data  Systems.  Maintenance  information  is  inadequate  at 
all  echelons  of  equipment  support.  At  the  unit  level,  technicians  lack  the 
information  which  would  help  them  do  a  better  job  (e.g.,  recent  equipment 
maintenance  history),  and  supervisors  lack  the  information  (e.g.,  jobs 
required  and  resources  available)  to  schedule  maintenance  work  efficiently. 

At  the  equipment-management  level  (Systems  Commands  and  field  activities), 
managers  must  rely  on  3-M  data  which  are  not  always  accurate  and  not  designed 
to  identify  causes  of  maintenance  problems.  The  Navy  lacks  an  aggressive  data 
automation  plan  designed  to  provide  information  feedback  to  the  technician, 
thus  assuring  self-interest  in  proper  maintenance  job  reporting. 

j.  Technical  Manuals.  While  the  Navy  has  made  substantial  progress 
with  improving  technical  manual  acquisition  and  support  through  adopting  a 
tailored  contract  requirements  concept  and  automating  distribution  management, 
serious  problems  remain.  The  chief  problems  are  incomplete,  inaccurate,  or 
out-of-date  information  so  that  technicians  must  rely  on  other  sources.  The 
Navy  Technical  Information  Presentation  System  (NTIPS),  and  its  predecessor 
R&D  program  were  intended  to  eliminate  those  problems,  but  it  has  grown  into  a 
very  ambitious  project  which  has  caused  delays  in  applying  automation  to  the 
generation,  distribution,  and  control  of  technical  information. 

k.  Supply  Support.  The  Navy's  stockage  policy  for  spare  parts  is  a 
demand-based  policy;  i.e.,  spares  and  repair  parts  are  authorized  for 
shipboard  stockage  only  if  they  meet  certain  minimum  demand  criteria.  With 
increasing  system  complexity  (increased  parts  count),  such  a  policy  becomes 
increasingly  suboptimal  as  the  probability  that  any  particular  part  will  fail 
decreases.  Switching  to  a  sparing-to-availability  methodology  would  increase 
A0  for  the  same  investment  in  spares  stockage  or  could  reduce  costs  without 
reducing  A0. 
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SECTION  IV 

KEY  ISSUES  FOR  MAINTENANCE  IMPROVEMENT 


OVERVIEW 

Inadequate  maintenance  performance  is  a  systemic  problem.  Piecemeal 
actions,  therefore,  offer  little  potential  for  improvement.  The  time  has  come 
for  a  broad  plan  of  action  across  functional  specialties  and  disciplines  to 
address  the  root-causes  of  the  problem.  The  assessment  of  the  problem  is  that 
such  a  plan  should  focus  on  the  following  key  issues  to  be  resolved  through 
further  R&D  when  necessary,  or  to  be  addressed  by  management  action: 

a.  Equipment  Design.  Design  characteristics  have  the  largest  potential 
impact  on  the  problem,  yet  an  effective  process  of  influencing  equipment 
design  for  maintainability  is  still  lacking. 

b.  Job  Design.  The  present  rate,  rating,  and  NEC  structure  exhibits 
excessive  overspecialization,  duplication,  and  a  serious  dichotomy  between 
training  for  jobs  and  training  for  advancement.  The  structure  is  not 
supportive  of  current  weapon  systems,  is  inflexible,  and  cannot  be  supported 
by  the  personnel  system.  Additionally,  the  Navy  lacks  an  effective  process 
for  job  design. 

c.  Organization  Design.  Due  to  combat  system  integration,  shipboard 
organization  is  no  longer  compatible  with  combat  system  architecture.  This 
causes  problems  with  divisional  responsibilities  for  diagnosing  failures  at 
the  system  level  and  in  subsystem  interfaces. 

d.  Maintenance  Philosophy.  The  current  maintenance  concepts  for 
electronic  equipment  installed  on  board  surface  ships  are  based  on  a 
maintenance  philosophy  which  is  inconsistent  with  operational  requirements  and 
cannot  be  supported  through  the  current  supply  system.  The  Navy  has  become 
aware  of  this  and  is  in  the  process  of  evaluating  alternative  maintenance 
policies.  Because  of  poor  coordination,  however,  actions  taken  in  other 
areas,  such  as  ATE,  are  preempting  the  evaluation. 

e.  Training  Philosophy.  The  current  training  philosophy  for  combat 
system  technicians  is  based  on  front-end  training.  Not  only  is  the  current 
approach  limited  in  effectiveness  and  efficiency,  it  is  also  demotivating  for 
many  trainees  and,  in  principle,  wasteful  of  scarce  training  resources. 

f.  Instructional  System.  Implementation  of  Instructional  Systems 
Development  (ISD)  has  resulted  in  impoverished  training:  course  content  lacks 
validity;  presentation  is  hampered  by  inadequate  training  equipment  for 
hands-on  training;  the  pass/fail  criterion  is  a  phony  "mastery"  concept. 
Apparently,  the  Navy  does  not  recognize  that  the  adopted  ISD  procedures  are 
designed  only  for  proceduralized  task  instruction,  not  for  development  of 
cognitive  skills  like  troubleshooting,  which  is  the  most  critical  job 
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c.  Combat  system  maintenance  concepts  will  continue  in  the  same 
direction  currently  exhibited: 

(1)  Attach  Submarines.  Improvements  in  reliability  through  fault- 
tolerant  design  will  permit  meeting  60-day  mission  requirements  with  minimum 
maintenance.  Host  of  the  corrective  maintenance  shill  requirements  will  be  at 
the  "I"-level,  augmented  by  Direct  Fleet  Support.  "C’-level  shill 
requirements  will  depend  on  the  extent  to  which  the  Navy  is  forced  to  tahe 
shortcuts  in  design  or  cuts  in  program  funding.  Keeping  a  small  number  of 
expert  technicians  aboard  would  reduce  design  complexity  and  lower  acquisition 
costs. 


(2)  Surface  Combatants.  Improvements  in  maintenance  diagnostics 
through  system  design  for  testability  will  permit  continued  reliance  on  a 
two-level  maintenance  concept  without  interfering  with  self-sufficiency.  The 
concept  may  fail,  however,  if  the  Navy  does  not  follow  through  with:  (1) 
increased  front-end  investment  to  achieve  the  level  of  testability  necessary 
to  eliminate  unnecessary  removals,  and  (2)  replacement  of  the  present 
demand-supported  stochage  policy  with  one  more  supportive  of  availability/ 
sustainability  requirements.  Corrective  maintenance  shill  requirements  will 
be  distributed  among  "0"-level  (fault  isolation  and  module  replacement),  some 
"I"-level  (screening  for  NFR  modules  and  repairing  selected  modules),  and 
"D"-level  (module  repair  and  rebuild  of  and  items). 

(3)  Tactical  Air.  Technology  will  permit  a  reduction  or 
elimination  of  "I"-level  avionics  repair  and  the  associated  ATE  aboard  the 
carrier.  The  exception,  however,  will  be  high-density  VHSIC/VLSI  boards, 
which,  for  reasons  of  affordability,  may  have  to  be  repaired  aboard  the 
carrier.  If  the  Navy  revises  its  traditional  maintenance  concept  accordingly, 
Fleet  maintenance  requirements  will  decline,  with  shill  requirements 
distributed  similarly  to  those  for  surface  combatants. 

d.  Job-tash  requirements,  both  for  operators  and  maintainers  of  combat 
systems,  will  increasingly  demand  cognitive  shills.  Tashs  that  can  be 
proceduralized  and  taught  as  such  will  constitute  an  increasingly  smaller 
portion.  More  attention  to  proper  job  design  and  a  total  change  in  training 
philosophy  will  be  required  if  the  Navy  is  to  meet  these  requirements. 
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ASSESSMENT.  History  may  repeat  itself.  In  the  late  1960s,  BIT  and  automated 
maintenance  diagnostics  were  adopted  as  the  solution  to  prevalent  personnel 
and  training  problems,  but  the  needed  testing  technology  was  not  available. 

The  consequences  became  all  too  apparent  in  the  mid-1970s.  If  the  clock  could 
be  turned  back,  the  testing  technology  currently  available  could  be  applied 
with  great  success  to  the  systems  developed  in  that  past  era.  Some  systems 
now  entering  the  Fleet  have  been  able  to  take  advantage  of  advances  in  testing 
technology  thanks  to  their  long  development  times.  Many  systems  have  not. 

Now,  the  presumption  of  many  individuals  is  that  the  testability  problem 
has  been  licked.  But,  it  is  often  forgotten  that,  in  the  meantime,  combat 
system  technology  has  not  stood  still.  The  gap  between  today's  testing 
technology  and  prime  equipment  technology  currently  on  the  drawing  boards  is 
as  large  as,  if  not  larger,  than  it  was  in  the  late  1960s.  The  consequences 
in  terms  of  maintenance  performance  and  A0  of  these  systems,  when  deployed 
by  the  late  1980s,  could  be  the  same  story  all  over  again. 

To  avoid  this  result,  the  following  steps  are  required: 

a.  R&D  efforts  to  advance  testing  technology,  including  the  tools  to 
apply  it,  must  be  accelerated  to  catch  up  with  the  next  generation  of  weapon 
system  technology. 

b.  Technology  insertion  demonstration  programs  are  necessary  to 
develop,  enhance,  and  verify  the  tools  required  to  achieve  testability. 

c.  The  rate  of  technology  advances  proposed  for  new  weapon  systems  must 
be  slowed  down,  pending  closure  of  the  testing  technology  gap.  Because  this 
is  unrealistic,  the  alternative  is  to  provide  for  a  planned  maturation  program 
during  the  production/operation  phase,  keeping  program  resources  (funding, 
people,  support  systems)  together  for  as  long  as  necessary  to  make  the  weapon 
system  supportable. 

SUMMARY 

The  projections  of  future  trends  in  manpower  and  personnel,  given  the 
Navy's  planned  build-up,  and  in  equipment  maintenance  requirements,  given  the 
direction  of  technology,  may  be  summarized  as  follows: 

a.  Requirements  for  experienced  electronics  technicians  will  increase, 
but  the  inventory  of  career  technicians  will  exhibit  an  increasing  deficit. 
Critical  skill  shortages  will  be  unavoidable,  unless  the  Navy  implements 
drastic  changes  in  personnel  and  manpower  procurement  policies. 

b.  First-term  personnel  must  be  equipped  to  perform  a  portion  of  the 
corrective  maintenance  workload  in  order  to  compensate  for  petty  officer 
shortages.  Improvements  in  training  and  aiding  are  necessary  to  achieve  this 
result. 
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Fault-tolerant  design  clearly  offers  significant  potential,  not  only  for 
improving  system  A0,  but  also  for  reducing  or  eliminating  “0"-level 
maintenance.  This  is,  essentially,  an  expensive  approach  to  solve  the 
maintenance  problem  and,  with  the  present  state-of-the-art,  not  cost-effective 
except  in  special  circumstances.  With  future  advances  in  this  technology,  the 
cost-effectiveness  ratio  can  be  expected  to  change,  so  that  this  approach  may 
become  increasingly  practical  in  the  1990s  for  mission-essential  subsystems. 

It  may  even  become  a  necessity  by  that  time,  if  a  breakthrough  in  VHSIC 
testing  technology  does  not  come  to  pass. 

ARTIFICIAL  INTELLIGENCE.  The  original  focus  of  AI  was  on  modeling  human 
cognition  in  some  limited  domain.  While  knowledge  representation  remains  the 
most  active  AI  research  area,  many  AI  practitioners  shifted  their  focus  in  the 
1970s  to  knowledge  engineering  applications.  Numerous  "expert  systems" 
(knowledge-based,  rule-based,  or  model-based)  and  associated  support  systems 
(relational  data  base  management,  inference,  and  interface  subsystems)  have 
been  developed  during  the  1970s,  mostly  for  experimental  purposes,  but  some 
for  operational  use.  Applications  of  AI  to  system  design  or  design  for 
testability  are,  at  the  present  time,  totally  experimental  but  are  believed  to 
have  promising  potential.  For  example,  two  applications  reported  in  the 
literature  are  KBVLSI  and  CRITTER.  The  former  is  an  experimental, 
knowledge-based  system  developed  by  Xerox  and  Stanford  University  to  aid  in 
VLSI  designs;  the  latter,  developed  at  Rutgers  University,  can  be  used  to 
check  the  correctness  and  robustness  of  digital  designs,  including  signal  and 
timing  margins,  using  heuristic  reasoning.  There  are  numerous  other 
application  programs  under  development.  The  OSD-IDA  R&M  Study  recommended 
that  AI  techniques  be  applied  to  VLSI/VHSIC  design  for  fault  tolerance  and 
testability,  and  that  such  an  RAD  project  be  incorporated  in  the  VHSIC  Phase  2 
program. 72 

Recently,  AI  has  received  increased  emphasis  by  DoD.  It  is  included  as  a 
key  technology  area  in  the  new  "intelligent  super  computer"  program  to  be 
launched  shortly  by  Defense  Advanced  Research  Projects  Agency  (DARPA),  who 
projects  a  cost  of  $1  billion  during  the  remainder  of  the  1980s.  This 
program,  which  encompasses  a  wide  spectrum  of  technologies,  is  currently  under 
review  by  a  special  task  force  of  the  Defense  Science  Board  to  recommend 
priorities.  A  final  report  of  this  review  is  scheduled  for  the  first  quarter 
of  1984. 

Any  assessment  of  the  potential  impact  of  AI  on  testability  would  be 
speculation.  AI  applications  to  maintenance  aiding  would  appear  to  be  more 
obvious  and  immediate.  Section  V  will  return  to  that  subject. 


72IDA  Document  No.  D-28,  Artificial  Intelligence  Applications  to 
Maintenance ,  April  1983  (Draft). 
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b.  Failures  can  occur  in  dedicated  spares  but  go  undetected  until  the 
spare  is  required  to  substitute  for  a  failed  active  component. 

c.  Latent  faults  tend  to  cause  a  continuous  "flip-flopping"  between 
active  and  spare  components. 

d.  A  failure  in  the  small  number  of  circuits  responsible  for 
reconfiguration  is  fatal. 

A  new  concept,  known  as  continuously  reconfiguring,  is  designed  to 
eliminate  these  shortcomings.  The  architecture  consists  of  a  pool  of 
autonomous  microprocessors,  both  active  units  and  nondedicated  spares, 
connected  by  a  common  set  of  data  buses.  Any  processor  is  able  to  perform  any 
task,  at  any  time,  with  task  assignments  continously  redistributed  among  all, 
using  a  concept  of  autonomous  control.  The  concept  is  under  development  and 
evaluation  by  the  Air  Force  Flight  Dynamics  Laboratory,  but  will  require  new 
bus  standards  (more  flexible  than  MIL-STD-1553B)  to  become  operational. 
Moreover,  there  is  currently  some  doubt  about  the  impact  of  this  architecture 
on  the  effectiveness  of  fault  detection. 7* 

Apart  from  multiprocessor  computer  applications,  fault-tolerant 
architecture  appears  to  be  in  its  infancy.  R&D  efforts  required  to  make  it 
possible  include  the  following  (as  identified  in  the  Air  Force's  PAVE  PILLAR 
program,  which  is  an  integrated  avionics  architecture  concept  study): 

a.  AI  technology  to  discriminate  true  faults  from  false  alarms, 

b.  adjustable  tolerances  for  fault  isolation, 

c.  improved  CAD  tools  to  integrate  testability  and  functional 
requirements, 

d.  new  sensors  and  associated  diagnostic  techniques, 

e.  testability  analysis  and  marginal  testing  techniques  for  design 
verification, 

f.  testing  techniques  for  diagnosing  cables  and  connectors, 

g.  "Smart"  BIT  strategies  and  algorithms  for  determining  accuracy  and 
stability  requirements  for  BIT, 

h.  new  high-speed  bus  architectures, 

i.  exploration  and  evaluation  of  alternative  fault-tolerant 
architectures  (depending  on  amounts  and  types  of  chip-level  BIT  available  from 
VHSIC)  through  simulation. 


71Richard  A.  Gunkel,  Gunkel  Consulting,  personal  communication,  1983. 
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maintenance  skill  requirements  (due  to  the  up-front  emphasis  on  testability, 
with  up  to  20  percent  of  real  estate  on  the  chip  reserved  for  self-test 
circuitry).  Apart  from  the  testability  design  problems  discussed  previously 
(Design  Automation),  it  is  believed  that  this  expectation  may  come  only  partly 
true  in  the  case  of  high-density  boards  for  the  simple  reason  of 
affordability,  as  explained  next. 

A  very  enlightening  study  of  this  topic  was  conducted  at  the  Air  Force 
Institute  of  Technology.7®  The  study  examined  the  various  elements  of 
life-cycle  cost  of  an  airborne  synthetic  aperture  radar  digital  processor 
based  on  VHSIC  technology.  The  study  found  that  of  all  the  variables 
influencing  life-cycle  cost,  wafer  size  and  on-line  testability  have  the  most 
profound  impact.  The  finding  with  regard  to  wafer  size  is  of  interest  because 
it  would  explain  why  some  electronics  companies  put  more  emphasis  on 
wafer-scale  integration  technology  than  on  the  philosophy  pursued  by  the  VHSIC 
program.  The  finding  with  regard  to  on-line  testability  could  have  a  major 
impact  on  the  maintenance  concept  for  VHSIC  technology  insertion  in  avionics 
systems,  such  as  planned  for  the  F-18  and  F-14.  While  the  plan  is  to  have  a 
two-level  maintenance  concept  ("0" -level  replacing  the  VHSIC  board  based  on 
BIT  indications,  and  "D"-level  repairing  the  board  through  chip  replacement), 
the  study  concludes  that  a  ten-fold  reduction  in  life-cycle  cost  would  be 
achieved  by  doing  the  repair  on-line  (i.e.,  aboard  the  aircraft  without 
removing  the  board). 

The  interpretation  of  that  study  is  that  the  Navy  may  need  to  reconsider 
its  planned  maintenance  concepts  for  VHSIC  technology.  Even  if  the  BIT 
performs  perfectly,  support  costs  may  drive  the  Navy  to  a  more  forward- 
oriented  support  concept  for  complex  VHSIC  modules  (comprising,  say,  100  ICs); 
i.e.,  shipboard  repair  of  those  modules  by  chip  replacement  vice  replacing  the 
module  with  a  spare  and  evacuating  it  to  depot. 

FAULT-TOLERANT  ARCHITECTURE.  Fault-tolerant  architecture,  not  to  be  confused 
with  design  for  "graceful  degradation,"  is  one  basic  approach  to  eliminating 
maintenance  at  the  "0"-level.  This  approach  has  been  standard  practice  at 
NASA  for  spacecraft.  As  previously  indicated,  it  is  also  the  approach  planned 
for  the  SUBACS  program.  Successful  implementation  of  the  approach,  however, 
relies  on  near-perfect  BIT  for  automatic  reconfiguration  of  the  system.  The 
approach  has  been  successfully  applied  in  multiprocessor  systems,  but  the 
standard  architecture  used  in  the  past  (centralized  control  of  reconfiguration 
and  stand-by  spares  to  be  activated  for  specific  computational  tasks  upon 
failure  of  specific  active  components)  exhibited  certain  drawbacks: 

a.  The  time  required  to  transfer  tasks  from  a  failed  processor  to  a 
spare  can  cause  transients  in  the  output  of  that  task. 


70Harold  W.  Carter,  "A  Preliminary  Investigation  of  the  Life  Cycle 
Costs  of  a  Digital  Processor  Using  Very  High  Speed  Integrated  Circuits,"  Air 
Force  Journal  of  Logistics  (Fall  1981),  pp.  9-17. 
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TABLE  5.  VHSIC  PROGRAM  STRUCTURE 


PHASE /DURATION 

GOALS/PRODUCTS 

BUDGET 

PHASE  0 

«wrch  -  Novrabtr  1980 

Prograa  definition  Including  detailed  concept*, 
deelgn  approach,  and  plana.  Product*  include 
architecture  and  daaign  approach**  aelacted  to 
lmplaaent  chip*  with  1.25  aieron  and  aubalcron 
ala*  device*. 

(Hiue  Contractor*) 

S10.5H 

PHASE  1 

M lay  1981  -  April  1984 

Daaign,  develop,  aad  pilot-produce  1.25  aieron 
chips  with  a  5x10**  gate-Rs/ca2  functional 
throughput  rate.  Develop  subayetaa  brass- 
board*.  Demonstrate  feasibility  of  design 
tools  and  simulation  aids.  Develop  aubalcron 
chip  technology.  Key  features  Include  BIT 
technology. 

(Six  Contractors) 

$  167. BH 

PHASE  2 

1984  -  Beyond 

Daaonatrate  Phase  1  brasaboarda  in  operational 
systaaa  (1.25  alcron  technology  insertion). 

Develop  technology  for  yield  enhancement. 

Develop  Integrated  Design  Autonation  Systsa. 
Develop  and  pilot-produce  aubalcron  chips 
with  10 1  *  gate-Ha/ca2  functional  throughput 
rat*. 

1984  -  1986:  $75M 

1984  -  1988  Estimate:  S400K 

PHASE  J 

1980  -  1986 

Develop  technology  supportive  of  Phases  1  >  2: 
architecture,  lithography,  design  automation, 
materials,  packaging  techniques,  fabrication, 
testing,  standardisation,  and  other  problem 

areas. 

(31  Contractors  so  far) 

1980  -  1984:  S50M 
(59  contracts  awarded) 

1980  -  1986:  $60M 

into  existing  operational  systems  or  those  under  development,  and  to  move  more 
cautiously  toward  submicron  goals. 

The  projected  benefits  from  VHSIC  technology  insertion  are  lower 
life-cycle  costs  and  increased  systems  performance,  availability,  and 
effectiveness.  A  preliminary  investigation  conducted  by  the  OSO-IDA  RAM  Study 
clearly  confirms  this  potential.69  For  example,  the  particular  cases 
examined  (paper  studies)  show  mean  time  between  failures  improvements  by 
factors  ranging  from  2.3  to  6.2,  acquisition  costs  reduced  by  a  factor  of  3, 
and  support  costs  reduced  by  a  factor  of  10.  Specifically,  with  regard  to 
maintenance,  the  expectation  is  that  VHSIC  technology  insertion  will  lower 
maintenance  requirements  (due  to  much  increased  reliability)  as  well  as 


69IDA  Document  No.  D-42,  VHSIC  Working  Group  Report,  April  1983  (Draft). 
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accordingly.®8  The  bottom  line  of  the  integrated  diagnostics  concept,  as 
discussed  in  Section  III,  is  that,  for  the  first  time,  a  precise  estimate  will 
be  available  of  the  "cognitive  load"  on  the  maintainer;  or,  vice  versa,  that 
this  "cognitive  load"  can  be  addressed  as  a  quantitative  constraint  in  the 
design  process.  The  associated  measures  have  meaning  for  the  design  engineer, 
can  be  tested  and  validated,  and  can  be  evaluated  through  operational  testing, 
if  properly  defined. 

JOB  DESIGN 

The  present  personnel  and  billet  structure  by  rate,  rating,  and  NEC  code, 
has  evolved  more  in  support  of  administrative  needs  than  through  a  planned 
effort  to  optimize  training  and  personnel  utilization.  The  structure  exhibits 
excessive  overspecialization,  proliferation  of  subspecialties,  duplication, 
and  a  serious  dichotomy  between  training  for  jobs  and  training  for 
advancement.  A  Navy  study  from  the  mid-1970s  summarizes  the  problems  as 
follows : 

a.  The  Study  Group  found  no  major  problems  in  the  availability  of 
qualified  people  or  the  demands  to  be  placed  upon  them  in  the  future  Navy. 
Where  the  main  problem  does  lie,  however,  is  in  redesigning  the  Navy  enlisted 
occupational  classification  system  (NEOCS)  so  that  it  will  maintain  the  supply 
of  skilled  people  where  they  are  needed.  The  Study  Group  defined  the  major 
problem  areas  of  NEOCS  as  follows: 

(1)  The  system  does  not  adequately  identify,  classify,  group,  or 
structure  skill  requirements  compatible  with,  or  supportive  of,  existing 
weapon  systems,  existing  logistics  systems,  and  their  respective  subsystems. 

(2)  NEOCS  does  not  provide  the  necessary  flexibility  in  the 
transfer  of  skills  to  accommodate  technological  change. 

(3)  NEOCS  promotes  overlapping  and  duplicating  classification  of 
occupational  fields. 

(4)  NEOCS  cannot  support  a  "One  Navy"  concept  employing  the 
cross-utilization  of  skills  in  different  commands. 


88Vice  Admira1.  Travers,  Vice  C1  , ef  of  Naval  Material,  in  his  keynote 
address  to  the  AUTOTESTCON  '81,  Orlando,  Florida,  October  1981.  Rear  Admiral 
Grich,  Vice  Commander  NAVELEX,  "The  Navy  Perspective  of  Testability,” 
conference  on  Design  for  Testability,  Arlington,  Virginia,  November  1981. 
Actually,  the  Navy's  Testing  Technology  program  dates  back  to  1977  when  it  was 
initiated  by  direction  of  the  Assistant  Secretary  of  the  Navy  for  RAD  in 
response  to  a  study  of  Fleet  problems  with  automatic  testing  (ATE  Ad  Hoc 
Working  Group,  "Report  on  Navy  Issues  Concerning  Automatic  Test,  Monitoring, 
and  Diagnostic  Systems  and  Equipment,"  February  1976). 
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(S)  As  a  system,  NEOCS  is  degrading  at  an  accelerating  rate.89 

The  recommendations  from  this  Study  Group  were  never  acted  upon  by  the 
Navy.  The  same  conclusions  still  apply  to  today's  lack,  of  proper  job  design 
for  the  technical  specialties  in  the  Navy.  The  NEOCS  Study  was  probably  the 
most  significant  personnel  and  training  study  conducted  in  recent  years;  but, 
ironically,  the  few  cognizant  personnel  interviewed  in  the  course  of  this 
study  either  did  not  know  about  the  NEOCS  study  or  implied  that  its 
recommendations  had  been  implemented. 

The  reasons  why  the  Navy  decided  not  to  follow-up  on  these 
recommendations  are  unknown.  Perhaps,  the  recommended  changes  from  the 
existing  manpower,  personnel,  and  training  structure  were  too  drastic  for  a 
tradition-prone  organization,  viz: 

a.  Consolidation  of  all  ratings  into  23  occupational  fields. 

b.  Elimination  of  the  linkage  between  skill  levels  and  pay  grades. 

c.  Deletion  of  petty  officer  designation  at  the  E-4  level. 

d.  Elimination  of  all  hardware-specific  NECs. 

e.  Restructuring  of  the  NEC  system  to  identify  skill  levels. 

f.  Revision  of  training  strategy  by  eliminating  the  front  loading  of 
training  costs,  introducing  a  distributed,  multilevel  training  approach,  and 
emphasizing  supervised  "hands-on"  training  at  Type  Commander  (TYCOM)  training 
activities. 

Nevertheless,  these  are  the  types  of  fundamental  changes  necessary  to  resolve 
the  maintenance  skill  proficiency  problem  within  affordable  cost  constraints. 
Additional  R&D  may  be  required  to  refine  the  results  of  the  NEOCS  Study,  but 
the  issue  is  more  one  of  management  action  than  one  requiring  basic  research. 

After  solving  the  issue  of  occupational  structure,  the  next  requirement 
is  development  and  implementation  of  a  more  effective  process  of  position 
analysis  or  design.  There  is  currently  no  single  document  or  data  base  which 
identifies  the  tasks,  skills,  and  knowledge  requirements  of  a  job  incumbent 
(occupant  of  a  billet).  Bits  and  pieces  of  the  information  are  found  in 
Personnel  Qualification  Standards  (PQSs),  Personnel  Advancement  Requirements 
(PARs),  Technical  Manuals  (TMs),  Maintenance  Requirements  Cards  (MRCs)  of  the 
Planned  Maintenance  System  (PMS),  and  Rate  Training  Manuals.  Consolidating 
all  this  information  into  a  single  reference  manual  was  one  of  the  major 
efforts  for  CSIP,  the  program  initiated  by  COMNAVSURFLANT  in  1982  to  improve 


89Chief  of  Naval  Personnel,  Navy  Enlisted  Occupational  Classification 
System  (NEOCS)  Study.  Vol.  II  (15  January  1974),  pp.  85-86. 
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the  proficiency  of  combat  system  technicians  through  diagnostic  testing  and 
remedial  or  refresher  training  (see  Section  II).  A  similar  effort  is 
currently  underway  at  CNET.  The  Naval  Enlisted  Professional  Development 
Information  System  (NEPDIS)  effort  is  comprehensive  and  described 
elsewhere. 90  While  it  will  provide  the  job-task,  data  needed  for  position 
analysis,  a  process  for  designing  positions  in  an  integrated  fashion  with  job 
aids  and  training  does  not  currently  exist.  The  variables  in  question  are 
known  from  the  literature,  but  attempts  to  improve  upon  position  design  often 
solve  one  set  of  problems  at  the  expense  of  another  set.9*  Further  RfiJ)  is 
required  to  bring  the  state  of  knowledge  in  job  design  further  along  toward 
application.  One  basic  aspect,  for  example,  that  has  not  received  rigorous 
analysis  is  the  trade-off  between  combining  or  not  combining  operator  and 
maintainer  tasks  into  the  same  job  position  and  the  associated  implications 
for  training  requirements,  skill  development,  job  satisfaction,  and 
proficiency. 

ORGANIZATION  DESIGN 

Closely  related  to  the  issue  of  job  design  is  that  of  organizational 
effectiveness.  This  term  is  normally  associated  with  the  theory  of 
organizational  development  in  behavioral  or  social  sciences.  Organizational 
development  is  very  well  addressed  by  the  Navy's  Human  Resources  Management 
Program  which  is  contributing,  indirectly,  to  improving  job  performance.  This 
section  is  addressed  at  an  entirely  different  issue,  that  of  shipboard 
organizational  structure  and  its  potential  impact  on  maintenance  effectiveness 

Traditionally,  weapon  subsystems  have  been  independent  entities  supported 
by  rating-specific  divisions  within  shipboard  organizations.  With  the  trend 
toward  computer-based  integration  of  subsystems  into  combat  systems,  a 
dichotomy  evolved  between  system  architecture  and  organizational  structure, 
causing  problems  with  assigning  responsibilities  for  diagnosing  system 
failures  to  the  appropriate  subsystem  interfaces.  The  Bureau  of  Naval 
Personnel  developed  in  the  early  1970s  the  "combat  system  technician"  concept 
to  deal  with  this  problem.92  The  concept  was  tested  aboard  a  number  of 
pilot  ships  by  reorganizing  billets  concerned  with  electronics  maintenance 
within  the  Operations  and  Weapons  Departments  into  a  single  Combat  System 
Department,  including  a  System  Test  Officer  billet  and  a  Ship's  Electronics 


"Thomas  H.  Ansbro,  An  Overview  of  the  Naval  Enlisted  Professional 
Development  Information  System  (NEPDIS).  Technical  Report  No.  122  (Orlando, 
Florida:  Training  Analysis  and  Evaluation  Group,  April  1982). 

91Theodore  J.  Post,  "Job  Design,  Performance  Aids,  and  Training,"  paper 
presented  at  The  Human  Factors  Society  Annual  Meeting,  October  1977. 

92Burea  of  Naval  Personnel,  Combat  System  Personnel  Training  and 
Management  Plan  (Washington,  D.C.:  Bureau  of  Naval  Personnel,  1  November 
1971). 
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Readiness  Team  (SERT)  to  which  senior  petty  officers  were  assigned  as 
collateral  duty  for  system-level  testing  and  casualty  diagnosis.  The 
experimental  test  had  serious  limitations  (i.e.,  training  was  not  changed, 
just  the  organizational  structure)  which  resulted  in  the  ambivalent  finding 
that  .  .  objective  measures  of  electronics  maintenance  performance  failed 
to  demonstrate  that  improved  maintenance  resulted  from  implementing  the  combat 
system  organizational  structure  onboard  pilot  ships."  The  conclusion  from 
this  evaluation  effort  was  as  follows: 

The  problem  concerning  electronics  readiness  of  Navy 
combatant  ships  is  not  yet  resolved.  The  centralized 
administration  of  electronics  maintenance  will  assist  in, 
but  is  not  sufficient  for,  resolution  of  this  problem. 

Factors  beyond  ship-level  management  of  technical 
personnel  contribute  to  the  lack  of  electronics  readiness 
of  Navy  ships.93 

The  current  status  of  this  effort  is  "in  limbo."  Only  some  pieces  of  the 
organizational  combat  system  technician  concept  have  been  implemented.  For 
example,  most  surface  combatants  have  a  SERT,  formally  or  informally,  but 
their  function  is  limited  to  identifying  skill  deficiencies  and  training 
requirements  to  meet  readiness  inspections;  they  are  not  responsible,  by 
standard  operating  procedure,  for  system-level  diagnosis  of  casualties.  The 
training  aspects  of  the  concept  were  never  implemented.  Clearly,  additional 
RAD  effort  is  necessary  to  bring  the  concept  to  fruitful  implementation. 

It  is  noteworthy  that  Naval  aviation  has  been  more  successful  in  adapting 
its  maintenance  organization  to  the  fundamental  issue  of  system-level 
diagnosis;  since  the  late  1960s,  both  the  Squadron  Manning  Document  and  the 
Naval  Aviation  Maintenance  Program  (Office  of  the  Chief  of  Naval  Operations 
(OPNAV)  Instruction  4790.2)  identify  a  Troubleshooters  Branch  in  the  Line 
Division  of  the  Organizational  Level  Maintenance  Department.  This  Branch  is 
manned  by  senior  petty  officers  from  the  various  ratings  for  system-level 
troubleshooting,  for  some  aircraft  on  a  full-time  basis  (e.g.,  F-14A) ,  for 
other  aircraft  (e.g.,  S-3A)  as  additional  duty  during  Flight  Quarters  (i.e., 
when  aircraft  are  launched  or  recovered)  by  petty  officers  from  the  various 
organizational  shops.  There  is  a  clear  parallel  between  the  Troubleshooters 
Branch  in  aviation  maintenance  and  the  SERT  in  ship  or  submarine  combat  system 
maintenance,  but  the  latter  was  never  fully  implemented. 

MAINTENANCE  PHILOSOPHY 

The  maintenance  concept  of  a  weapon  system  or  end  item  of  equipment 
defines  the  conditions  under  which  it  is  maintained,  the  tasks  (what,  when, 


93Lloyd  S.  Standlee  and  Del  H.  Sass,  Combat  System  Personnel  Training 
and  Management  Evaluation:  Final  Report,  Technical  Report  TR  77-17  (San 
Diego:  Navy  Personnel  Research  and  Development  Center,  February  1977).  This 
report  summarizes  the  results  of  15  separate  studies  devoted  to  this  topic. 
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and  how}  performed  at  each  maintenance  level,  and  the  skill  categories 
(rate/rating/NEC)  to  which  the  tasks  are  allocated.  The  maintenance  concept 
and  equipment  design  characteristics  are  closely  related  factors;  they 
mutually  influence  each  other  to  such  an  extent  that  the  maintenance  concept 
is  often  viewed  as  a  design  characteristic.  Other  factors,  however,  also 
influence  the  maintenance  concept;  these  include  the  operational  concept  (in 
Navy  terminology,  the  Plan  for  Use),  established  maintenance  policy,  and  the 
existing  maintenance  structure.  At  the  root  of  the  latter  factors  is  an 
overall,  generic  view  of  how  maintenance  is,  or  should  be,  performed:  a 
factor  referred  to  as  maintenance  philosophy.  That,  ultimately,  drives 
maintenance  policies  or  doctrine,  the  maintenance  concepts  espoused  for  new 
equipment,  and  consequently,  the  equipment  design  characteristics  related  to 
maintenance  support.  As  a  result,  the  maintenance  philosophy  has  a  pervasive 
influence  on  many  maintenance-related  factors,  including  job-task  requirements 
and  training. 

To  explain  how  changes  in  maintenance  philosophy  have  contributed  to  poor 
maintenance  performance,  as  evidenced  by  low  A0,  it  is  necessary  to  go  back 
to  the  early  1970s.  At  that  time,  the  Navy  reevaluated  established 
maintenance  policy  in  view  of  changes  in  electronics  technology  and  shortages 
of  resources  (especially  manpower)  required  to  meet  existing  and  projected 
maintenance  support  needs.  The  outcome  was  a  revised,  uniform  maintenance 
policy  for  electronics  equipment,  eliminating  on-equipment  repairs  from  the 
"0"-level  by  means  of  modular  design  (permitting  removal/replacement,  vice 
repair  of  failed  modules),  moving  module  repairs  to  higher  maintenance  levels 
(favoring  "D"-level  over  *'I"-level  on  the  basis  of  economics),  and  increasing 
the  dependence  on  "I"-level  support  (establishing  restricted  availabilities 
for  "I"-level  maintenance  in  order  to  extend  the  period  between  major 
overhauls)  (NAVMAT  Instruction  4790.19,  June  1973).  This  policy  was 
subsequently  firmly  established  in  Navy  maintenance  doctrine  (NAVMAT 
Instruction  4700. 4B,  "Levels  of  Equipment  Maintenance,"  August  1974). 

Adoption  and  implementation  of  this  modular  concept  ignored  two 
fundamental  conditions  to  make  it  effective:  (1)  testability  design  to 
localize  system  failures  to  the  removable  modules,  and  (2)  on-board  stockage 
of  spare  modules.  Neither  condition  was  met.  As  a  result,  the  first 
deficiency  resulted  in  a  high  NFR  rate  which,  in  the  absence  of  any  screening 
capability  at  the  "0"-level,  depleted  the  supply  system.  That,  in  combination 
with  the  second  deficiency,  resulted  in  increasing  CASREPs  for  parts.  To 
counter  the  untoward  impact  on  readiness,  the  Fleet  developed  the  miniature/ 
microminiature  (2-M)  electronic  repair  program.  This  program  consisted  of 
training  courses  for  circuit  board  repair,  installation  of  precision  soldering 
equipment  aboard  selected  surface  combatants,  and  authorization  to  perform 
emergency  repairs  of  circuit  boards. 9*  The  program  was  strictly  a 


94NAVSEA  Instruction  4790.17,  "Miniature/Microminiature  (2-H) 
Electronic  Repair  Program,"  May  1980;  and  NAVMAT  Instruction  4790. 22A, 
"Miniature  Electronic  Repair  Program,"  February  1981. 
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"band-aid";  was  implemented  without  authorized  parts,  schematics  of  circuit 
boards,  or  test  equipment;  and,  because  of  the  uncertain  influence  on  system 
integrity,  resulted  in  as  many  problems  as  it  solved. 

By  the  early  1980s,  the  situation  had  become  bad  enough  for  the  Fleet  to 
confront  the  CNO  and  the  Chief  of  Naval  Material  with  requests  to  change  the 
maintenance  philosophy,  authorizing  the  "0"-level  to  perform  module  repair  in 
a  more  organized  fashion  (providing  schematics  and  piece  parts),  and  the 
"I"-level  to  screen  all  modules  using  ATE,  returning  good  boards  and  repairing 
defective  boards.  The  situation  was  summed  up  by  CINCLANTFLT  as  follows: 

To  throw  away  a  $450  board  because  of  a  defective  $3.95  IC 
is  foolish.  To  reduce  combat  capability  of  a  $100  million 
ship  because  the  closest  replacement  module  is  undergoing 
repair  at  a  depot  is  criminal,  but  both  examples  occur  on 
a  regular  basis  because  of  today's  depot-only  repair 
concept.95 

Caught  between  peacetime  cost-effectiveness  considerations  and  the 
Fleet's  demands  for  self-sufficiency,  the  Navy  decided  to  study  its  modular 
philosophy  once  more.  The  result  of  this  study  consisted  of  identifying  four 
alternatives:96 

a.  Provide  the  Fleet  with  modular  repair  capability. 

b.  Provide  the  Fleet  with  modular  screening  capability. 

c.  Provide  the  Fleet  with  selective  modular  repair  capability. 

d.  Provide  SIMAs  with  screening  and  repair  capability. 

It  did  not  recommend  one  over  the  other. 

Since  no  decisions  have  been  made,  NAVSEA  is  proceeding  with  two  programs 
designed  to  test  and  evaluate  the  last  two  alternatives.  Under  the  Support 
and  Test  Equipment  Engineering  Program  (STEEP),  pilot  facilities  have  already 
been  established  at  SIMAs  in  Norfolk  and  San  Diego  to  test  and  repair  circuit 
boarus  using  ATE.  A  total  of  68  STEEP  sites  are  planned  under  this  program: 


95Commander-in-Chief ,  U.S.  Atlantic  Fleet,  Memorandum,  Subject:  Draft 
Navy  Training  Plan  for  Radar  Display  and  Distribution  Systems,  November  1981. 
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a. 

AD/AR/AS : 

28 

b. 

CV/CVT: 

14 

c. 

LHA/LCC: 

7 

d. 

NOTU : 

10 

e. 

SIMA: 

5 

f. 

SRF : 

4 

The  second  program  underway  is  a  pilot  program  of  shipboard  testing  and 
repairing  of  selected,  digital  modules  from  the  Nk  86  GFCS  aboard  selected 
ships  of  the  DD-963  Class.  This  program  was  initiated  in  February  1981  at  the 
direction  of  CNO.  The  test  and  evaluation  plan  was  approved  in  September 
1981,  and  an  18-month  observation  period  began  in  August  1982  after 
installation  of  test  equipment,  technical  documentation,  piece-parts,  2-M 
repair  station,  and  training  courses.  Based  on  a  progress  report  reviewed,  it 
appears  that  limited  funding  and  a  too-small  sample  of  circuit  board  failures 
may  threaten  the  validity  of  test  results.97  In  any  event,  the  program  is 
to  provide  a  cost-benefit  analysis  and  assess  the  ramifications  in  training, 
documentation,  support  equipment,  spares,  and  personnel  requirements.  It  is 
scheduled  for  completion  in  FY84. 

In  sum,  the  current  situation  demonstrates  a  lack  of  consistency  between 
policy  and  actions,  and  highlights  a  dire  need  for  better  coordination,  as 
observed  by  a  recent,  informative  study  conducted  at  the  Naval  Postgraduate 
School:98 


97GFCS  Hk  86  On  Board  Nodule  Repair  Pilot  Program,  Quarterly  Status 
Report  (February  1983). 

98LT  N.  E.  Brown,  USN,  Improving  The  Electronic  Repair  Capabilities  in 
The  Fleet,"  (N.  S.  Thesis,  Naval  Postgraduate  School,  March  1982). 
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a.  Current  NAVMAT  maintenance  policy  is  not  consistent  with  CNO  ship 
maintenance  policy.  As  the  latter  is  in  accord  with  DoD  maintenance  policy, 
NAVMAT  must  change  its  policy." 

b.  The  evident  trend  toward  increasing  ship's  force  corrective 
maintenance  capabilities  will  require  central  direction  of  maintenance  policy, 
coordination  of  2-M  and  ATE  programs,  improved  supply  support,  improved 
training,  and  consideration  of  other  factors  than  just  cost  in  level-of-repair 
analyses . 

TRAINING  PHILOSOPHY 

Navy  technical  training  is  front-loaded.  Host  formal  technical  training 
is  given  prior  to  any  job  experience,  while  little  skill-enhancement  training 
is  available  for  career  personnel  (other  than  self-study  training  materials). 
Training  durations  for  the  13  different  ratings  responsible  for  electronics 
maintenance  vary  by  track,  but'  extend  to  approximately  one  year  for  the 
longest  tracks  {not  counting  Recruit  Training).  The  only  other  training  that 
technicians  might  receive  during  their  Navy  career  consists  of  additional  "C" 
School  courses  when  they  are  assigned  to  maintain  equipment  not  covered  in 
previous  "C"  School  courses.  (For  the  avionics  ratings,  skill  progression 
training  after  "A"  School  is  provided  by  the  Fleet  Readiness  Squadron  and 
Naval  Air  Maintenance  Training  Detachment  (NAMTRADET),  similar  to,  but  not  a 
"C"  School.  One  key  difference  is  that  trainees  in  those  ratings  (AQ,  AT,  AX) 
receive  "practical  job  training"  (i.e.,  supervised  OJT)  on  operational 
equipment  in  an  environment  resembling  the  operational  environment;  nC" 

Schools  do  not  provide  this). 

The  high  cost  and  limited  effectiveness  of  this  approach  to  technical 
skill  training  has  been  a  subject  of  concern  for  many  years.  First,  the 
training  is  ineffective  because  most  graduates  do  not  possess  the  skills 


"OPNAV  Instruction  A700.7F,  "Maintenance  of  Ships,"  September  1981, 
includes  corrective  maintenance  as  an  "0"-level  responsibility: 
"Organizational  (Shipboard)  level  maintenance  is  that  corrective  and 
preventive  maintenance  accomplished  by  the  ship's  crew.  The  individual  ship 
shall  be  maintenance  self-sufficient  to  the  degree  achievable  within  manpower 
and  facility  constraints."  In  contrast,  NAVMAT  Instruction  4700. 4B  limits 
"0"-level  responsibility  to:  "...  inspecting,  servicing,  lubricating, 
adjusting  and  replacing  of  parts,  minor  assemblies  and  sub-assemblies."  DoD 
guidance  on  maintenance  support  consists  of  the  following:  DoD  Directive 
4151.1  states  that  military  units  should  perform  direct  maintenance  (where 
possible)  and  prescribes  the  role  for  contractor  personnel;  DoD  Instruction 
4151.12  assigns  responsibility  for  developing  maintenance  support  concepts  to 
Maintenance  Engineering;  DoD  Directive  4151.16  states  that  maintenance  should 
be  performed  near  the  point  of  generation  and  oriented  to  weapon  systems;  and 
DoD  directive  5000.39  encourages  the  use  of  innovative  support  concepts  to 
improve  system  readiness/support  costs. 
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required  to  develop  job  proficiency  through  on-the-job  experience.  Second,  it 
is  inefficient  because  the  benefits  to  the  Navy  are  not  in  consonance  with  the 
high,  upfront  investment.  Contributing  causes  include  high  personnel 
turnover,  a  weak  linkage  between  training  content  and  job-task  requirements, 
and  a  personnel  utilization  which  results  in  losing  most  of  the  skills  and 
knowledge  acquired  in  training  (i.e.,  during  the  first  three  to  six  months  on 
their  first  duty  station,  school  graduates  are  typically  assigned  to  mess 
duty).  Third,  the  training  approach  is  wasteful  since  over  20  percent  of 
high-aptitude  students  wash  out  in  Basic  Electricity  and  Electronics  (BEE) 
School. 100,101,102  jhe  jiavy  cannot  afford  losing  these  people.  The  cause 
of  this  high  attrition  rate  is  not  a  lack  of  student  aptitude,  but  the 
inflexibility  of  the  present  computer-managed  instruction  (CHI)  course  which 
does  not  adapt  teaching  style  to  individual  cognitive  styles  and  abilities, 
additionally,  because  a  needs  analysis  was  never  done  to  determine  BEE-course 
content,  the  course  lacks  any  demonstrable  relationship  with  the  Navy 
technician's  job. 

For  an  outsider,  it  is  very  hard  to  understand  how  this  approach  to 
training  came  about.  No  one  else,  outside  the  U.S.  Armed  Services,  trains 
this  way.  Our  NATO  allies  —  specifically  the  British,  Germans,  and  Dutch  — 
put  more  emphasis  on  upfront  OJT  to  familiarize  trainees  with  the  equipment  in 
the  operational  environment.  They  apply  a  multilevel,  distributed-training 
concept  (i.e.,  distributed  over  the  individual’s  career),  fitting  the  training 
to  job-task  requirements,  as  well  as  providing  retention  incentives.  In  many 
cases,  factory  training  rounds  off  the  training  program  for  senior  technicians 
in  the  career  force.  Why  the  U.S.  Navy  has  not  yet  changed  its  training 
philosophy  may  be  due  to  two  persistent,  common  beliefs  which  had  been  exposed 
as  myths  many  years  ago:103 

a.  Myth  7:  The  Military  Services  can  take  a  high  school  graduate  and 
turn  him  into  a  field  service  technician  within  a  few  months. 

b.  Myth  9:  On-the-job  training  is  a  good  way  to  teach  troubleshooting. 


l°°David  C.  Atwater  and  Norman  M.  Abrahams,  Evaluation  of  Alternative 
ASVAB  Composites  for  Selected  Navy  Technical  Schools.  NPRDC  TR-80-15  (San 
Diego:  Navy  Personnel  Research  and  Development  Center,  February  1980). 

101J.  Sachar,  Macy  Abrams,  and  Christine  Buckley,  Relating  Performance 
in  Basic  Electricity  and  Electronics  and  "A"  Schools.  NPRDC  TR-79-2  (San 
Diego:  Navy  Personnel  Research  and  Development  Center,  October  1978). 

102CINCLANTFLT  Trip  Report  dated  February  1982. 

103Nicholas  A.  Bond,  Jr.,  "Some  Persistent  Myths  About  Military 
Electronics  Maintenance,"  Human  Factors.  Vol.  12,  No.  3  (June  1970),  pp. 
241-252. 
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In  essence,  the  Navy  does  not  have  a  training  "system"  designed  to  ensure 
that  technicians  acquire  the  skills  and  knowledge  required  for  proficient  job 
performance  along  a  career  continum.  There  is  no  adequate  occupational  data 
base  (see  "Job  Design").  There  is  no  career-development  planning.  And  there 
is  a  sense  of  lack  of  direction  as  one  recently  retired  CNET  staff  member 
observes : 1^4 

So  we  go  blithely  on  our  way,  with  "A"  schools  basing 
their  curricula  on  one  set  of  occupational  data,  some 
teaching  basic  skills,  others  teaching  to  specific 
equipment;  "C"  schools  using  the  rate  training  manuals 
which  use  still  another  set  of  data  and  have  no  connection 
with  what  the  "A"  schools  teach;  the  personnel 
qualification  standards  program  plodding  along  on  its  own 
data  base;  and  so  on.  With  no  system,  there  is  nothing  to 
manage  in  the  true  sense,  creating  confused  sailors  and 
unhappy  fleet  commanders. 

Additionally,  the  presumption  that  Fleet  training  will  fill  the  voids  in 
formal  school  training,  either  through  formal  OJT  or  "F"  Schools,  is 
unrealistic  because  required  training  materials  and  other  resources  are  not 
available.  Fleet  training  also  is  hampered  by  the  fact  that,  since  the 
mid-1970s,  no  new  PQSs  have  been  written  for  maintenance  tasks  outside  the 
Navy  Air  community.  Only  the  avionics  technicians  receive  the  benefit  of  a 
short  period  of  formal  OJT.  This  was  one  of  the  reasons  why  the  NEOCS  Study 
recommended  institutionalizing  TYCOM  training  activities  with  formally 
specified  (and  resourced)  training  responsibilities  for  OJT,  diagnostic 
testing  of  skill  deficiencies,  and  refresher  training  programs. 

In  1977,  the  NAVPERSRANDCEN  attempted  to  fill  this  void  by  developing  an 
integrated  personnel  systems  concept  showing  the  interdependencies  among 
personnel,  job  aids,  job  design,  career  planning,  training,  technical  data, 
and  equipment  design  characteristics.  This  signalled  the  start  of  the 
Enlisted  Personnel  Individualized  Career  System  (EPICS),  an  R&D  project  to 
develop  a  JPA-based  integrated  personnel  system  model,  implement  the  model  in 
a  Fleet  setting,  and  conduct  a  longitudinal  test  and  evaluation  program.  The 
specific  features  of  the  EPICS  model  are:*05 


104CAPT  Worth  Scanland,  USN  (Ret.),  "Training  Sailormen:  Why  Isn't  It 
Working,"  Naval  Institute  Proceedings  (October  1983),  pp.  54-57. 

10^A  more  detailed  description  of  EPICS  is  provided  by:  R.  E. 

Blanchard  and  R.  J.  Smillie,  Integrated  Personnel  Systems  Approach:  The 
Enlisted  Personnel  Individualized  Career  System  (EPICS)  Model.  NPRDC  TN  80-14 
(San  Diego:  Navy  Personnel  Research  and  Development  Center,  May  1980). 
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a.  Place  recruits  directly  from  recruit  training  into  a  shipboard 
apprentice  program  and  provide  job  indoctrination.  Require  EPICS  sailors  to 
mess  cook,  compartment  clean,  and  to  perform  other  facilities  maintenance  as 
scheduled  part  of  the  program. 

b.  Defer  and  distribute  shore-based  technical  training  over  a  four-year 
enlistment  and  provide  such  training  in  two  episodes:  Equipment  Technician 
Training  at  the  12th  month  and  System  Technician  Training  at  the  24th  month. 

c.  Conduct  job-design  studies  and  define  three  skill  levels  for  the 
first  four-year  enlistment,  identified  as  Apprentice  Technician,  Equipment 
Technician,  and  System  Technician. 

d.  Develop  fully  proceduralized  aids  to  meet  all  technical  job 
requirements  during  the  first  year  (Apprentice  Technician),  with  partially 
proceduralized  aids  to  meet  job  requirements  during  the  second  year  (Equipment 
Technician).  Develop  advanced  troubleshooting  aids  for  use  during  remaining 
first-term  enlistment  as  System  Technician. 

e.  Invest  in  individual  shore-based  training  as  a  function  of 
demonstrated  interest,  performance,  level  of  shipboard  adaptation,  completion 
of  all  program  and  military  prerequisites,  and  recommendation  of  commanding 
officer. 


f.  Provide  shipboard  instruction  by  means  of  self-teaching  exportable 
packages  with  narrative,  summary,  and  programmed  instruction  including 
subject-scored  module  tests  and  supervisor-scored  comprehensive  tests. 

g.  Develop  technical  competency  through  building  blocks  involving 
practical  job  experience,  formal  OJT,  instructional  modules  provided  for  each 
skill  level,  and  shore-based,  resident  school  training. 

h.  Provide  a  personnel  career  system  with  supporting  organization, 
explicit  advancement  paths,  integrated  training  episodes  and  job  performance 
aids,  specified  time  frames  and  prerequisites,  assessment  points,  promotion 
points,  and  career  decision  points.  Consider  also  such  factors  as  new 
personnel  assimilation  into  Fleet  units,  demotivation,  turbulence  and 
attrition  contributors. 

In  October  1979,  the  Deputy  Chief  for  Naval  Operations  (Manpower, 
Personnel,  and  Training)  approved  the  project  to  proceed  to  pilot 
implementation  for  the  NATO  SEASPARROW  Surface  Missile  System  and  to  conduct  a 
four-year  longitudinal  study  to  test  and  evaluate  the  cost  and  benefits  of 
EPICS  vis-a-vis  conventional  training.  Thisf decision  received  endorsement  by 
the  Fleet  in  1980.  The  study,  conducted  by  the  NAVPERSRANDCEN,  is  being 
monitored  by  the  EPICS  Steering  Group  with  membership  from  OPNAV  (OP-01, 

OP-10 ,  OP-11,  OP-13,  OP-03),  Naval  Military  Personnel  Command,  CNET,  Navy 
Recruiting  Command,  and  NAVMAT  (MAT  08D) .  Recruitment  of  EPICS  sailors  began 
in  July  1980;  shipboard  data  collection  began  September  1980  and  will  continue 
into  FY85.  The  Fleet  test  and  evaluation  is  scheduled  for  completion 
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September  1985;  the  cost-effectiveness  analysis  by  September  1986;  and 
publication  of  the  Final  Report,  December  1986. 

EPICS  is,  by  far,  the  most  important  R&D  project  undertaken  by  the  Navy 
to  improve  maintenance  performance.  It  has  far-reaching  implications  and  is 
expected  to  demonstrate  substantial  benefits  and  cost  savings  over  the 
conventional  training  approach.  The  interim  reports  are  most  favorable,  and 
EPICS  has  been  universally  acclaimed  as  the  model  for  all  Services  by  experts 
who  have  scent  their  professional  careers  in  addressing  task  performance 
problems . 

Further  research  will  be  required  to  ensure  successful  implementation  of 
EPICS  for  all  combat  system  technicians  (and,  possibly,  other  occupational 
groups  not  of  interest  to  the  present  study).  This  effort  relates  to  other 
topics  discussed  in  this  section:  job  design  (revamping  the  occupational 
structure),  instructional  system  development  (course  content),  personnel 
system  (need  to  develop  individual  career  ladders  and  ensure  that  personnel 
return  to  training  when  scheduled),  and  reconsideration  of  some  ideas  espoused 
by  the  NEOCS  Study  (TYCOM  Training  Activities). 

INSTRUCTIONAL  SYSTEM 

In  the  late  1950s,  training  technologies  began  to  apply  systems  analysis 
techniques  to  training  course  development,  resulting  in  training  design 
manuals  for  use  by  laymen  with  little  or  no  experience  in  training  technology 
or  educational  psychology.  According  to  one  study,  over  one  hundred  such 
manuals  were  published  between  1960  and  1975. In  the  mid-1970s,  DoD, 
through  its  Interservice  Training  Review  Organization,  adopted  and  prescribed, 
for  use  by  all  Military  Departments,  a  standard,  proceduralized  approach,  the 
Interservice  Procedures  for  Instructional  Systems  Development,  commonly 
referred  to  as  ISu.108  CNET  adapted  this  approach  to  the  needs  of  CNET 
formal  schools  and  promulgated  a  condensed  manual  (Naval  Education  and 
Training  (NAVEDTRA)  110,  Procedures  for  Instructional  Systems  Development)  in 
1978  for  the  development  of  new,  or  revision  of  existing,  instructional 
programs.  Submarine  training  systems,  however,  are  governed  by  NAVSEA  OD45519. 


106NSIA/DoD,  Proceedings  of  the  Second  Annual  Conference  on  Personnel 
and  Training  Factors  in  Systems  Effectiveness  (San  Antonio,  Texas:  May  1982). 

107Melvin  D.  Montemerlo  and  Michael  E.  Tennyson,  Instructional  System 
Development:  Conceptual  Analysis  and  Comprehensive  Bibliography.  Technical 
Report  IH-257  (Orlando,  Florida:  Naval  Training  Equipment  Center,  February 
1976) . 

108r.  k,  Branson,  et  al.,  Interservice  Procedures  for  Instructional 
Systems  Development,  TRADOC  Pamphlet  350-30  and  NAVEDTRA  106A,  4  Vols.  (August 
1975). 
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ISO  is  commonly  perceived  as  a  rational  approach  to  job-relevant, 
task-oriented  training.  In  reality,  there  are  basic  problems  with  the 
approach  itself  as  well  as  its  implementation: 

a.  The  ISD  manual  is  a  guideline  showing  what  must  be  done,  with  little 
guidance  in  how  to  do  it.  In  the  hands  of  novices,  it  is  a  dangerous  tool  as 
evidenced  by  the  serious  problems  experienced  DoD-wide  with  ISD 

implementation. 

b.  The  ISD  model,  as  adopted  and  implemented  by  the  Services,  was 
designed  exclusively  for  "proceduralized"  tasks,  not  for  cognitive  tasks  such 
as  troubleshooting.  Original  plans  to  include  such  tasks  in  the  final  version 
were  aborted  when  the  preliminary  version  of  ISD  was  adopted  for  use;  only  the 
Army  is  in  the  process  of  developing  an  "extended  task-analysis  procedure"  for 
nonproceduralized  tasks. 110 

c.  The  ISD  approach  uses  behavioral  objectives  and  criterion-referenced 
measures  in  training,  but  is  based  on  a  simplistic  view  of  complex  performance 
as  a  stimulus-response  chain  of  simple,  discrete  steps.  Performance  is 
described  in  terms  of  external  cues  and  feedback,  involving  measurable 
responses,  steps  to  be  performed  according  to  prespecified  sequences,  and 
capable  of  being  communicated  in  the  form  of  written  material.*11  The  ISD 
model  thus  omits  any  task  requirements  which  cannot  be  couched  in  these  terms, 
including  the  "headwork"  associated  with  integrating  tasks  in  job  performance. 

d.  Aside  from  these  inherent  limitations,  the  Navy's  implementation  of 
ISD,  in  practice,  suffers  from  the  following  limitations: 

(1)  The  guidelines  for  selecting  the  instructional  setting  in  the 
ISD-analysis  phase  are  designed  to  allocate  training  to  resident  schools  only 
as  a  last  resort.  Application  of  the  guide  results  in  allocating  training  in 
priority  order  to  OJT,  on-board  training  with  self-instruction  materials,  "F" 
Schools,  or  Resident  ("A"  or  "C")  Schools.  Yet,  self-instruction  materials 


109R.  Vineberg  and  J.  N.  Joyner,  Instructional  System  Development  in 
the  Armed  Services:  Methodology  and  Application,  Technical  Report  80-1 
(Alexandria,  Virginia:  Human  Resources  Research  Organization,  January  1980). 

110Dr.  Robert  K.  Branson,  Director  and  Principal  Investigator, 
Interservice  Procedures  for  ISD,  Center  for  Educational  Technology,  personal 
communication  (Tallahassee:  Florida  State  University,  July  1983).  The  Army 
manual  in  question  is  TRADOC  Pamphlet  351-4,  Chapter  9. 

111Gary  A.  Klein,  Phenomenological  Approach  To  Training.  AFHRL-TR-77-42 
(Dayton,  Ohio:  Air  Force  Human  Resources  Laboratory,  Wright-Patterson  Air 
Force  Base,  August  1977). 
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have  not  been  developed  in  the  past.  CNET  only  recently  announced  that 
production  of  on-board  training  packages  has  been  initiated. 

(2)  Criterion-referenced  testing  is  a  basic  aspect  of  ISD.  But  the 
pass/fail  criterion  for  most  courses  is  getting  80  or  90  percent  of  the  test 
items  correct.  Because  training  objectives  are  not  stated  in  a  hierarchy  of 
importance  or  difficulty,  tests  sample  among  all  objectives  so  that  an  80  or 
90  percent  criterion  may  be  achieved  without  meeting  the  most  difficult  or 
most  important  objectives.  This  is  why  this  aspect  of  ISD  implementation  is 
referred  to  as  a  phony  mastery  concept. 

(3)  An  inherent  concept  of  the  ISD  model  is  the  need  for  internal 
and  external  evaluation  to  control  the  quality  (training  effectiveness)  of  a 
training  course.  The  two  primary  feedback  channels  for  external  evaluation 
are  the  CNET  questionnaire  distributed  to  supervisory  Fleet  personnel  and  the 
NAVMAT  technical  audit.  Both  appear  limited  in  effectiveness.  The 
NAVPERSRANDCEN  spent  several  years  developing  and  testing  a  more  detailed, 
standardized  approach  to  job  performance  monitoring,  but  this  effort  was 
aborted  by  the  Navy.  As  a  result,  there  is  currently  no  meaningful  feedback 
mechanisms  for  evaluating  training. 

(4)  There  is  no  linkage  between  PAR  and  the  terminal  objectives  of 
ISD-based  courses,  other  than  by  happenstance.  There  is  no  single, 
coordinated  job-task  analysis  done  in  the  Navy;  everyone  in  need  of  job-task 
information  goes  their  own  way:  training  course  developers,  training 
equipment  developers,  technical  manual  writers,  PQS  and  PAR  developers,  etc. 
The  root-cause  is  that  the  single,  integrated  system,  LSA,  has  never  been 
implemented  as  conceived. 

Concern  about  ISD  and  the  way  it  has  been  implemented  by  the  Navy  is 
growing.  The  Fleet,  as  a  whole,  finds  the  training  product  —  i.e.,  the 
qualifications  of  school  graduates  —  unacceptable.  At  the  top-levels  in  the 
Fleet,  the  root-cause  is  attributed  to  the  emphasis  on  procedure  training  at 
the  exclusion  of  cognitive  training.113  At  the  worker-level,  teams 
responsible  for  ISD-syllabus  development  are  deviating  from  the  formal 
procedures  in  recognition  of  the  voids  in  ISD  and  slipping  cognitive  detail 
back  into  the  syllabus.  But  these  attempts  are  uncoordinated,  are  subject  to 
the  experience  of  the  team  members,  and  are  insufficient  to  divert  the  overall 
ISD  thrust;  they  also  become  less  substantial  over  time  as  ISD  teams  begin  to 


113VADH  James  A.  Sagerholm,  USN,  "CNET:  Where  We  Are  And  Where  We  Are 
Going,"  U.S.  Naval  Institute  Proceedings  (October  1983),  pp.  59-64. 

113James  J.  Regan,  former  Technical  Director  NAVPERSRANDCEN,  personal 
communication,  June  1983. 
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include  ISD-trained  personnel  who  do  not  know  better.  And  educators  begin  to 
wonder  whether  the  elimination  of  cognitive  training  was  a  wise  idea  after 
all.114 

This  enumeration  of  critical  observations  on  the  implementation  of  ISD 
must  not  be  interpreted  as  an  indictment  of  the  ISD  concept;  i.e.,  the  need  to 
base  training  curricula  on  a  thorough  task  analysis,  develop  terminal  and 
enabling  objectives  based  on  that  analysis,  determine  the  best  mix  of  training 
presentation  media,  and  establish  a  closed-loop  feedback  system  to  evaluate 
training  effectiveness.  Rather,  the  institutional  shortcomings  in 
implementing  ISD  are  at  fault,  including  the  naive  presumption  that  the 
current  ISD  manual  is  all  that  is  needed.  While  training  managers  may  believe 
that  ISD  solves  their  "training  problem,"  ISD  has  served  to  highlight  the 
inadequacy  of  current  scientific  knowledge  about  training.  ISD,  as  currently 
institutionalized,  is  incomplete,  and  does  not  provide  the  answer  as  to  what 
kind  or  level  of  cognitive  training  is  required  to  ensure  job  proficiency. 
Further  R&D  is  necessary  to  advance  the  state-of-the-art;  the  results  must  be 
incorporated  into  ISD  to  evolve  the  current  inadequate  process  into  one  that 
ensures  training  content  validity. 

The  amount  and  kind  of  theory  required  in  technical  writing  has  always 
been  a  matter  of  conjecture.  An  ISD-based  course  may  or  may  not  change  the 
theory  content  of  the  course  it  is  replacing.  There  are  no  objective  data 
comparing  course  content  before  and  after  a  course  was  "ISD’d."  Thus,  some  of 
the  Fleet  observations  reported  above  may  be  based  on  perception,  not  fact. 
Past  studies  indicate  that  the  extent  of  theory  in  Navy  technical-training 
courses  is  already  limited,  regardless  of  ISD.  For  example,  a  recent  survey 
of  course  materials  at  21  "A"  Schools  (excluding  BEE)  and  77  "C"  and  "F" 
Schools  (with  12  of  the  98  courses  "ISD'd")  shows  the  following:115 


114J.  R.  Brock,  “Simulation  in  Maintenance  Training:  Now  That  I’ve 
Thrown  Out  the  Bath  Water,  Where  is  the  Dear  Baby?”,  paper  presented  at 
American  Educational  Research  Association,  New  York,  1977. 

115William  E.  Montague  and  Wallace  H.  Wulfeck,  II,  Navy  Personnel 
Research  and  Development  Center,  personal  communication,  June  1983  (report  in 
press).  The  survey  uses  the  Instructional  Quality  Inventory  approach  designed 
for  evaluating  instructional  programs  (see  NPRDC  SR  79-3,  November  1978). 

This  approach  classifies  each  training  objective,  test  item,  or  piece  of 
instruction  in  accordance  with  a  task/content  matrix.  Tasks  are  defined  as: 
remember,  use-unaided,  or  use-aided;  content  is  defined  in  terms  of  five 
types:  fact,  concept,  procedure,  rule,  and  principle.  Remembering  (through 
recall  or  recognition)  or  using  principles  presumes  understanding  of 
cause-effect  relationships  and  is,  therefore,  theory-based.  Remembering  or 
using  rules  also  presumes  some  understanding  as  it  pertains  to  a  sequence  of 
steps  applicable  across  different  situations  or  equipments. 
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a.  "A"  Schools.  Terminal  objectives  are  primarily  remembering  facts 
and  using  procedures. 

b.  “C"/"F"  Schools.  Terminal  objectives  are  primarily  using  procedures. 

c.  Average  Theory  Content.  About  12  percent  of  the  terminal  objectives 
relate  in  some  way  to  theory  or  understanding  (five  percent  are  principles, 
seven  percent  are  rules). 

Thus,  assuming  there  is  a  direct  relationship  between  syllabus  hours  and 
numbers  of  terminal  objectives  (an  assumption  which  is  not  necessarily  true), 
only  five  percent  of  the  course  content  of  current  training  syllabi, 
regardless  of  ISD,  is  devoted  strictly  to  theory.  Whether  this  is  too  much  or 
too  little  depends  on  the  training  objective.  If  the  training  objective  is  to 
teach  a  first-term  student  to  use  the  fully  proceduralized  manual  on  the  job, 
it  may  be  too  much.  If  the  objective  is  to  teach  him  expert  troubleshooting 
procedures,  it  may  be  too  little,  given  that  this  is  all  the  formal  school 
training  he  would  receive.  Development  of  troubleshooting  skills  requires 
more  than  learning  a  few* standard  rules  and  applying  them  in  free-play  on 
operational  equipment;  it  requires  a  thorough  understanding  of  theory  of 
operations,  functional  interdependencies,  and  symptom-cause  relationships  in 
order  to  form  a  "cognitive  map"  or  model  of  the  system  to  be  maintained.116 
In  the  absence  of  a  career-oriented  training  philosophy,  training  objectives 
do  not  differentiate  between  these  different  levels  of  job  requirements  so 
that  some  level  of  theory  is  invariably  taught,  but  too  much  for  the  lowest 
and  too  little  for  the  higher  skill  levels. 

Lack  of  troubleshooting  skills  is  the  chief  deficiency  noted  in  graduates 
from  formal  school  training  throughout  their  first-enlistment  term.  They  lack 
the  foundation  to  develop  this  skill  through  experience  on  their  own.  The 
training  establishment,  however,  insists  it  provides  training  in  the 
particular  skill.  The  explanation  probably  lies  not  with  ISD,  not  with 
low-aptitude  personnel  or  ability  deficiencies,  nor  with  inept  instructors, 
but  with  the  training  approach.  As  far  as  it  can  be  determined  (based  on 
interviews),  the  training  consists  of  teaching  the  use  of  procedures 
(sequences  of  fixed  steps)  with  the  aid  of  a  manual  for  a  small  number  of 
pre-identif ied  equipment  malfunctions  —  those  that  are  easy,  well-documented 
in  the  manual,  and  not  necessarily  representative  of  those  encountered  in  the 
operational  environment.  In  most  cases,  such  practical  troubleshooting 
training  consists  of  instructor  demonstrations  on  actual-equipment  trainers, 
not  student  hands-on  practice  (by  lack  of  simulators  designed  for  the 
purpose).  No  one  should  be  amazed  that  this  does  not  lead  to  a  proficient 


116Nicholas  A.  Bond  and  Douglas  M.  Towne,  Troubleshooting  Complex 
Equipment  in  the  Military  Services:  Research  and  Prospects.  Technical  Report 
No.  92  (Redondo  Beach:  University  of  Southern  California,  Behavioral 
Technology  Laboratories,  December  1979). 
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troubleshooter.  More  importantly,  this  type  of  training  approach  may  actually 
interfere  with  the  development  of  troubleshooting  proficiency  later  on.117 

In  sum,  the  DoD-wide  implementation  of  ISD  in  1975  was  intended  to 
improve  the  cost-effectiveness  of  resident  school  training  in  the  Services  by 
eliminating  training  not  relevant  to  the  job,  standardizing  the  process  for 
training  syllabus  development,  and  reducing  training  duration  and  support 
costs.  This  action  was  in  response  to  a  Congressional  mandate  in  1971,  when 
Congress,  for  the  first  time,  assumed  authorization  and  appropriation 
responsibility  for  training  as  a  separate  line  item  in  the  DoD  program  and 
budget.  This  action,  however,  had  the  opposite  effect.  The  ISD  model  adopted 
has  fundamental  limitations  which  are  not  recognized  by  the  decision  makers. 
The  support  costs  are  much  higher  than  anticipated.  While  training  durations 
have  been  reduced  (not  necessarily  due  to  ISD  itself),  the  net  effect  has  been 
that  costs  have  risen  and  effectiveness  declined.  The  frustrations 
encountered  with  ISD  implementation  are  very  well  expressed  by  a  Navy  officer 
responsible  for  the  development  and  evaluation  of  the  F-14  ISD  syllabi i1*® 

The  Navy  training  establishment  has  committed  the  future 
of  naval  aviation  to  ISD.  (This)  approach  to  training  has 
promised  increased  training  effectiveness,  greater 
standardization,  reduced  cost,  reduced  manpower,  and 
greater  flexibility  and  control.  It  has  produced  much 
less  effective  training  because  of  its  serious  omissions; 
it  has  produced  less  standardization  because  of  the  high 
cost  and  lag  time  inherent  in  media  update;  it  has 
dramatically  increased  training  costs;  it  has  made 
training  dependent  on  complex  and  unreliable  simulators; 
it  has  increased  the  manpower  requirements;  and  it 
produces  syllabi  that  are  less  flexible  and  less 
manageable  than  their  predecessors.  It  has 
systematically,  if  unintentionally,  destroyed  the 
traditional  foundations  of  naval  aviation.  The 
consequences  for  combat  effectiveness,  safety,  and  esprit 
de  corps  are  only  now  becoming  visible.  The  future 
implications  are  deeply  disturbing.  It  is  time  for  a 
complete  reexamination  of  the  direction  of  naval  aviation 


117Gary  A.  Klein,  Phenomenological  Approach  to  Training.  AFHRL-TR-77-42 
(Dayton,  Ohio:  Air  Force  Human  Resources  Laboratory,  Wright-Patterson  Air 
Force  Base,  August  1977). 

118CDR  Nicholas  R.  Criss,  III,  USN,  "Aviation  Training:  A  Systems 
Approach,"  U.S.  Naval  Institute  Proceedings  (October  1983),  pp.  102-106. 

While  his  comments  are,  primarily,  directed  at  flight-crew  training,  they 
pertain  as  well  to  maintenance  training.  His  criticism  of  flight  simulators 
is  not  so  much  an  ISD  issue  as  caused  by  the  limitations  of  the  F-14  training 
simulator. 
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training.  It  is  time  to  forsake  ISD  for  a  true  systems 
approach  that  encompasses  not  only  behavioral  tasks,  but 
also  the  entire  cognitive  framework  without  which  naval 
aviation  cannot  survive. 

It  is  not  suggested  that  ISD  be  abolished.  Rather,  the  Navy  should 
formulate  an  aggressive  R&D  program  to  fill  the  voids  and  develop  needed 
improvements  in  the  ISD  process.  Personnel  tasked  with  applying  ISD  should 
receive  the  requisite  training.  Additionally,  the  process  should  be 
coordinated  with  a  career-oriented,  distributed  training  "system"  to  improve 
the  linkage  between  job  requirements  and  training.  With  regard  to 
troubleshooting  skills,  there  is  an  obvious  need  for  a  more  comprehensive 
approach  to  teach  what  is  commonly  referred  to  as  logical  troubleshooting: 
understanding  the  theory  of  operations  of  an  equipment  family  to  acquire  a 
mental  model  of  failure  symptom-cause  relationships;  learning  the  approach  of 
hypothesis  formation,  selection,  and  testing;  and  acquiring  the  skill  to  apply 
that  approach  effectively,  including  proficient  use  of  test  equipment,  across 
a  broad  range  of  prime  equipment.  Unfortunately,  after  30  years  of  research, 
"educational  experts"  still  do  not  agree  on  the  most  effective  method  for 
teaching  troubleshooting  (problem-solving)  skills. The  R&D 
recommendations  (Section  V)  are  designed  to  help  resolve  this  issue. 

INDIVIDUAL  PROFICIENCY  ASSESSMENT 

The  Navy  has  an  elaborate  system  for  monitoring  the  "health"  of  its 
hardware  (3-M  system,  CASREP  system,  Material  Condition  Reporting  System). 
There  is  no  system  for  monitoring  the  job  proficiency  of  its  maintenance 
technicians.  (The  personnel  and  training  readiness  indicators  included  in  the 
UNITREP  system  refer  exclusively  to  operator  skills  and  the  fill  of  authorized 
billets  in  critical  rate/rating/NECs ,  not  to  the  job  proficiency  of 
maintenance  personnel.)  This  shows  where  the  real  priorities  lie,  even  though 
personnel  and  training  represent  a  large  portion  of  the  O&M  Navy  budget. 
Without  a  standardized  method  for  proficiency  assessment,  evaluation  of 
training  effectiveness  is  impossible.  This  also  inhibits  identifying 
shortcomings  in  technical  training  and  conflicts  with  the  whole  idea  of 
ISD-based  training.  The  current  controversy  between  the  Fleet  and  the 
training  establishment  is  a  clear  illustration  of  the  consequences.  A  more 
fundamental  implication  is  that  job  proficiency  may  be  perceived  by 
technicians  and  their  supervisors  as  a  low-priority  item:  people  normally 
behave  in  accordance  with  established  evaluation  systems,  paying  attention  to 
what  is  measured  or  monitored. 


119Gary  A.  Klein,  Improvement  of  Skills  for  Solving  Ill-Defined 
Problems ,  AFHRL-TR-78-31  (Dayton:  Air  Force  Human  Resources  Laboratory, 
Wr ight-Patterson  Air  Force  Base,  March  1979). 
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In  the  past,  the  Navy  has  spent  a  large  R&D  effort,  through  the 
NAVPERSRANDCEN ,  developing  a  personnel  proficiency  assessment  system 
(PPAS).120  This  effort  came  to  a  standstill,  reportedly  as  a  result  of  lack 
of  interest  by  the  Fleet,  which  viewed  it  as  yet  another  administrative 
burden.  In  the  meantime,  however,  the  issue  of  maintenance  skill  deficiencies 
has  gained  increasing  attention  and  visibility  at  the  top  echelons  in  the 
Fleet.  The  various  maintenance-training  improvement  programs  instituted  by 
all  Type  Commanders  to  identify  skill  deficiencies  and  determine  refresher, 
remedial,  or  skill-enhancement  training  requirements  have  already  been 
described.  These  programs  are  under-resourced,  uncoordinated,  duplicative, 
and  not  always  effective.  The  Navy  would  benefit  from  applying  the  technology 
gained  with  the  PPAS  R&D  program  to  these  maintenance-training  improvement 
programs  to  arrive  at  a  standardized,  Navy-wide,  performance-measurement 
system.  It  appears  that  the  Fleet  may  be  more  receptive  to  such  an  idea  than 
it  was  some  years  ago. 

Institution  of  such  a  system  appears  to  be  crucial  to  the  success  of  any 
attempts  to  improve  maintenance  performance.  The  resulting  feedback  is 
necessary  to  keep  formal  school  training  in  tune  with  Fleet  requirements. 

Only  for  strategic  systems  does  the  Navy  conduct  an  operational  program  for 
testing  the  proficiency  of  technicians  (this  system  is  known  as  the  personnel 
training  evaluation  program).  Such  a  system,  however,  is  not  a  luxury,  but  a 
necessity  for  all  Navy  technicians. 

PERSONNEL 

Among  the  personnel  factors,  personnel  availability  (fill  of  authorized 
Fleet  billets  by  rate/rating/NEC)  is  the  most  important  factor  influencing 
maintenance  performance.  The  factors  influencing  personnel  enlistment  and 
retention  have  been  previously  described  and  it  has  been  shown  that  changes  in 
personnel  policies  and  manpower  recruitment  are  necessary  to  improve  personnel 
availability  —  changes  that  will  become  increasingly  essential  with  the 
planned  Navy  build-up.  The  focus  here  is  on  three  related  personnel  issues: 
personnel  classification,  utilization,  and  management. 


120PPAS  is  described  in  numerous  NAVPERSRANDCEN  reports,  including 
tests  conducted  for  selected  ratings.  See,  for  example:  Adolph  V.  Anderson, 
et  al.,  A  Personnel  Readiness  Training  Program:  Final  Report.  NPRDC  TR  77-39 
(San  Diego:  Navy  Personnel  Research  and  Development  Center,  August  1977);  and 
T.  Athnos,  A  Performance  Proficiency  Assessment  System  for  Surface  Sonar 
Technicians:  Shipboard  Gathering  Procedures.  NPRDC  TR  80-16  (San  Diego:  Navy 
Personnel  Research  and  Development  Center,  February  1980);  and  Andrew  P. 
Chenzoff,  Dennis  L.  Scott,  and  Reid  P.  Joyce,  A  Performance  Proficiency 
Assessment  System  for  Surface  Sonar  Technicians:  Specifications  for  Simulated 
Troubleshooting  Tests.  NPRDC  TN  81-19  (San  Diego:  Navy  Personnel  Research  and 
Development  Center,  July  1981). 
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PERSONNEL  CLASSIFICATION.  Personnel  are  currently  classified  (assigned  to 
rating-specific  "A"  Schools)  on  the  basis  of  their  ASVAB  scores,  individual 
preferences,  recruiter  sales  techniques,  and  available  school  seats.  The 
system  implemented  in  June  1981  is  known  as  CLASP  (Classification  and 
Assignment  with  PRIDE,  where  the  latter  stands  for  personalized  recruiting  for 
immediate  and  delayed  enlistment).  CLASP  generates  a  recruit's  enlistment 
options  based  on  the  best  match  of  his  mental  group  ranking  and  the  complexity 
rankings  of  ratings  for  which  school  seats  are  available  in  a  one-month 
window;  the  system  also  accommodates  formal  or  unofficial  cutting  scores  on 
area  aptitude  composites.  Once  the  recruit  decides,  PRIDE  automatically 
reserves  a  school  seat  for  him. 

The  whole  process  very  much  depends  on  the  reliability  and  validity  of 
the  ASVAB,  a  test  with  several  limitations.  Congress  has  repeatedly  requested 
DoD  to  develop  better  test  instruments  more  predictive  of  job  performance 
(vice  training  performance)  than  the  present  ASVAB,  with  higher  differential 
validity  and  higher  reliability.  The  Defense  Manpower  Commission  has  made 
similar  recommendations .^^1  The  Navy  is  the  lead  Service  for  the 
development  of  computerized  adaptive  testing  (CAT)  as  one  response  to  this 
Congressional  mandate.  It  was  therefore  expected  that  the  CAT  project  would 
entail  introduction  of  other  tests  known  to  be  more  predictive  of  job 
performance.  This  is  not  the  case.  CAT  is  an  individualized,  interactive 
version  of  the  same  ASVAB  tests.  By  zeroing  in  on  the  individual  aptitude 
level,  rather  than  the  average  level  for  which  the  conventional  ASVAB  is 
designed,  CAT  will  improve  somewhat  the  validity  and  reliability  of  test 
results  at  less  cost  (testing  duration):  the  extent  of  guessing  by  test 
takers  will  be  more  uniformly  distributed  instead  of  being  dominant  at  the 
lower  aptitude  levels;  the  test  items  in  the  data  bank  are  carefully 
calibrated  in  terms  of  difficulty,  discriminative  power,  and  ease  of  guessing; 
and  test  time  is  expected  to  be  reduced  by  50  percent.  CAT  is  scheduled  for 
implementation  in  1985,  with  the  current  ASVAB  eliminated  by  1989. 

It  is  known  that  most  of  the  individual's  basic  abilities  which  influence 
training  and  job  performance,  are  not  measured  by  the  ASVAB.  The  extent  of 
classification  errors  caused  by  the  current,  inadequate  testing  approach  is 
unknown  by  lack  of  data  (e.g.,  type  1:  high  ASVAB  scores  but  low  job-related 
ability  scores;  type  2:  vice  versa).  What  is  known,  is  that  the  Navy  is 
losing  20  percent  of  high-aptitude  enrollments  in  BEE  School  (see  footnotes 
100,  101,  102)  by  its  inability  to  adapt  teaching  style  to  individual, 
cognitive  styles  because  the  latter  are  unknown  and  not  measured. 

The  ASVAB  test  requires  approximately  2  1/2  hours  to  complete.  Without 
extending  that  duration,  CAT  offers  the  potential  for  adding  tests  of 
cognitive  abilities  to  improve  the  person-job  match  for  combat  system 
technicians.  The  Navy  should  examine  the  extent  of  current  misclassif ication, 


i^lDefense  Manpower  Commission,  Defense  Manpower:  The  Keystone  of 
National  Security.  Report  to  the  President  and  the  Congress,  April  1976. 
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develop  ability  profiles  for  specific  task  clusters,  expand  classification 
standards  and  CAT  test  battery  accordingly,  and  adapt  teaching  style  to 
individual  abilities. 

PERSONNEL  UTILIZATION.  Two  examples  serve  to  demonstrate  that  the  ways  in 
which  personnel  are  currently  utilized  are  not  conducive  to  technical  skill 
development  and  growth.  One  is  the  use  of  school  graduates  for  general  duty 
details  (mess  cooking,  etc.)  during  the  first  three  to  six  months  on  their 
first  duty  station.  During  this  period,  much  of  what  was  learned  in  resident 
schools  was  forgotten  because  the  training  emphasis  is  on  rote  learning,  not 
understanding.  If  the  normal  flow  of  nondesignated  apprentices  (non-"A" 

School  eligibles)  is  insufficient  to  provide  the  necessary  manpower  for  Buch 
general  duty  details,  there  is  an  easy  way  to  increase  the  flow  but  avoid  the 
loss  of  training  investment  in  designated  strikers:  send  every  enlistee  first 
to  apprentice  training,  then  to  the  Fleet  for  OJT,  familiarization,  and 
general  duty;  and  return  all  "A"  School  eligibles  to  school  after  the  first 
six  months.  This  is  more  or  less  the  first  stage  of  the  EPICS  approach;  the 
increase  in  Permanent  Change  of  Station  costs  should  be  well  worth  the 
increase  in  training  effectiveness. 

The  second  example  is  that  shipboard  technicians  receive  little  hands-on 
experience  in  the  maintenance  tasks  for  which  they  are  responsible.  Most  of 
the  time  (roughly  75  percent)  is  spent  on  watch  standing  and  other  service 
diversions,  not  maintenance  job  performance.  Under  those  circumstances,  skill 
development  on  the  job  is  slow,  even  for  the  most  motivated  persons.  Without 
continuous  attention  by  supervisors  or  Fleet  training  activities  on  the  need 
for  refresher  training,  self-study,  and  hands-on  practice,  skills  may  actually 
degrade  over  time.  There  is  no  easy  answer  to  this  problem.  However,  if 
there  is  no  better  way  of  designing  the  jobs,  a  formalized  on-board  training 
system  for  maintenance  technicians  would  be  essential  to  keep  skills  from 
degrading.  As  described  earlier,  such  a  training  system  does  not  presently 
exist.  This  circumstance  also  highlights  the  need  for  periodic  diagnostic 
testing  (proficiency  assessment)  to  identify  skill  degradation  and  remedial/ 
refresher  training  needs.  In  the  absence  of  either,  most  technicians  cannot 
be  expected  ever  to  achieve  proficiency. 

PERSONNEL  MANAGEMENT.  On  the  surface,  the  Navy  appears  to  have  a  rigorous 
management  system  in  place  to  ensure  that:  (1)  personnel  are  assigned  where 
needed,  and  (2)  the  best  possible  overall  utilization  of  available  personnel 
is  attained.  In  reality,  however,  there  are  come  problems  which  inhibit 
achieving  these  objectives: 

a.  The  Navy  Manning  Plan  (NMP)  distributes  available  manpower  assets  by 
rate/rating  across  Navy-wide  authorized  billets  so  as  to  achieve  an  equal 
proportionate  fill  of  authorizations  in  all  units  not  accorded  manning 
priority.  A  non-linear  distribution  logic,  based  on  marginal  utility  rather 
than  percentage  fill,  would  result  in  a  more  effective  utilization  of 
available  manpower. 


91 


.1 

J 


-*M  --  -  *  -  *  *  -  *  -  -  «  -r  -  -*•  -  A  "v 


*  •  '  %  v  ”  •  -  *0  ■  **  J 

■  ~  J* .  -s  ."w.r  IkaJ 


NAVTRAEQUIPCEN  MDA903-81-C-0166-1 


b.  The  assignment  decisions  made  by  personnel  detailers  at  the  Navel 
Military  Personnel  Command  are  based  on  incomplete  personnel  data  (only  the 
primary  and  secondary  NEC),  incomplete  billet  data  (the  NMP  is  in  terms  of 
rate/rating  so  that  the  detailer  must  guess  the  NMP  allocation  among  different 
NECs  identified  by  OPNAV  Form  1000/2),  and  are  performed  manually  and 
sequentially.  The  use  of  an  optimal  assignment  system  would  improve  the 
person-to-billet  match. 

c.  The  data  on  OPNAV  Form  1000/2  and  the  Enlisted  Distribution 
Verification  Reports  (EDVRs)  are  inaccurate  due  to  lack  of  discipline, 
inadequate  training  of  administrative  personnel,  and  confusing  instructions/ 
regulations.  For  example,  a  recent  audit  showed  that  close  to  30  percent  of 
the  NECs  listed  on  OPNAV  Forms  1000/2  were  wrong,  and  36  percent  of  NECs  shown 
on  the  EDVRs  were  wrong. 

Clearly,  the  Navy  must  improve  personnel  management;  the  steps  required 
immediately  follow  from  the  above  findings.  On  a  more  fundamental  level,  the 
Navy  should  recognize  the  basic  conflict  between  its  highly  discriminate 
occupational  structure  (96  ratings,  1000  NECs)  and  the  ability  of  its 
personnel  system  to  support  the  level  of  discrimination.  Special 
"communities"  may  escape  that  conflict  through  "closed-loop"  detailing  (as 
practiced  in  the  submarine  community).  But  that  option  cannot  be  applied 
Navy-wide  for  all  ratings.  We  refer  again  to  the  NEOCS  Study  and  the  need  to 
revamp  the  Navy's  occupational  structure.  Pending  such  action,  "closed-loop" 
detailing  of  combat  system  technicians  might  be  the  only  answer. 

MAINTENANCE  DATA  SYSTEMS 

The  3-M  system  consists  of  two  major  subsystems,  the  Planned  Maintenance 
System  (PMS)  and  the  Maintenance  Data  System  (MDS).  PMS  is  used  for  the 
planning,  scheduling,  and  management  of  personnel  and  material  to  accomplish 
planned  (preventive)  maintenance.  MDS  is  used  for  recording  the  expenditure 
of  resources  (personnel  and  material)  associated  with  corrective  maintenance 
and  for  identifying  deferred  maintenance.  The  3-M  system  for  Naval  aviation 
is  more  detailed  and  more  vigorously  controlled  than  that  for  ships.  The 
summary  description  that  follows  pertains  to  the  3-M  system  as  implemented  in 
the  surface  and  subsurface  communities. 

Each  ship's  PMS  is  defined  by  Maintenance  Index  Pages  (MIPs)  and 
associated  MRCs  which  specify  for  each  equipment,  the  preventive  maintenance 
actions  required  and  their  frequency;  the  tasks  or  steps  associated  with  each 
action;  the  rate/rating/NEC  required;  and  estimated  man-hours.  PMS  is 
monitored  by  the  Type  Commanders  and  Systems  Commands.  Revisions  to  PMS, 
based  on  feedback  from  the  Fleet,  are  promulgated  through  the  Quarterly  Force 
Revision  (QFR)  program,  updating  MIPs/  MRCs  as  required. 


122Naval  Audit  Service  Capital  Region,  Review  of  Personnel  Detailing 
Functions .  Audit  Report  C37121,  February  1982. 
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Data  input  into  the  HDS  is  by  Maintenance  Data  Form  (OPNAV  Form  4790-2K, 
commonly  referred  to  as  the  "2K"  sheet).  These  forms  document  all  maintenance 
performed  or  required  beyond  routine  preventive  maintenance  defined  by  the 
PHS.  The  "2K"  sheets  thus  cover: 

a.  Completed  maintenance  actions,  including: 

(1)  All  reportable  corrective  maintenance  (not  all  equipment 
maintenance  is  reportable,  but  all  combat  system  maintenance  is). 

(2)  PMS  actions  for  which  the  MRC  specifies  use  of  repair  parts 
(all  parts  requisitions  require  "2K"  documentation,  which  is  one  reason  why 
all  maintenance  actions  requiring  parts  issue  by  the  supply  room  are 
documented,  and  many  other  maintenance  actions  are  not). 


reported. 


PMS  actions  requiring  meter  readings  or  tolerances  to  be 


(4)  Any  preventive  maintenance  other  than  PMS. 


b.  Deferred  maintenance  actions,  including: 

(1)  Corrective  maintenance  not  accomplished  due  to  operational 
priorities  or  need  for  outside  assistance. 

c.  Work  requests  for  outside  assistance. 

All  HDS  data  is  accumulated  in  a  central  Navy  data  base  by  the  Navy 
Maintenance  Support  Office.  Each  ship  accumulates  its  own  data  base  on 
deferred  maintenance  actions,  known  as  the  Current  Ship  Maintenance  Project 
(CSHP).  Maintenance  inspections  by  the  chain-of-command  focus  on  the 
performance  of  PMS  requirements  (the  CNO  goal  is  that  at  least  75  percent  be 
accomplished).  The  CSHP  serves  as  a  starting  point  for  the  development  of 
work  packages  when  ships  go  into  overhaul. 

Feeding  and  maintaining  the  above  systems  is  a  large  administrative 
effort,  but  only  a  small  part  of  the  total  shipboard  administrative  burden. 

The  Navy  initiated  the  Shipboard  Non-Tactical  ADP  Program  (SNAP)  to  alleviate 
this  burden.  The  program  is  managed  under  the  auspices  of  OPNAV  Instruction 
5320.16  and  directed  by  the  Fleet  Non-Tactical  ADP  Policy  Council.  It 
consists  of  two  major  efforts: 

a.  SNAP  I:  Replacing  the  existing  obsolete  AN/UYK-5  computers  aboard 
large  ships  and  at  shore  facilities. 

b.  SNAP  II:  Providing  smaller  ships  and  selected  shore  activities  with 
a  standard  ADP  system  compatible  with  SNAP  I. 
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The  hardware/software  specifications  developed  by  NAVSEA  in  accordance 
with  the  functional  specification  for  SNAP  II  cover  approximately  200 
functions;123  the  majority  {88  percent)  is  associated  with  supply  and 
maintenance  functions,  primarily  to  serve  the  information  needs  of  the  shore 
establishment.  Previous  studies  have  recommended  that  people-related 
applications  be  accorded  more  emphasis;12*  one  example  is  a  personnel 
training  management  system. 12^  But  the  present  direction  of  the  SNAP 
program  is  focused  on  automating  what  is  currently  done  manually,  eliminating 
manual  files  and  manual  generation  of  reports.  Installations  are  planned  for 
about  450  platforms,  spread  out  over  a  period  of  six  years,  starting  mid-1982. 

The  above  background  information  serves  to  highlight  one  key  issue: 
current  maintenance  data  systems,  manual  or  automated,  serve  primarily  the 
information  needs  of  the  shore  establishment;  left  out  are  the  needs  of  the 
maintenance  technician.  Specifically,  the  technician  is  not  provided  with  an 
up-to-date  maintenance  history  of  the  equipment  he  is  supporting;  nor  does  the 
technician  receive  feedback  on  the  components  removed  from  the  equipment  for 
checkout  and  repair  at  higher  echelons.  The  first  type  of  data  would  help 
identify  maintenance  characteristics  of  the  equipment  and  reduce  NFR  rate, 
mean-time-to-repair ,  and  awaiting  parts  delays.  The  second  type  of  data  would 
help  understand  performance  deficiencies  (i.e.,  maldiagnosed  faults)  and 
reduce  NFR  rate. 

The  Navy  must  recognize  this  information  void,  re-think  the  current 
orientation  of  SNAP  II,  reconsider  the  potential  advantages  of  distributed 
microprocessors  vice  SNAP  II  computers,  and  develop  a  data  automation  approach 
to  serve  the  needs  of  the  shipboard  technician.  Recent  experience  in  the  Air 
Force  has  shown  that  substantial  improvements  in  maintenance  performance  can 
result  from  simple  ADP  applications  for  providing  maintenance  information  to 
aircraft  maintainers.  An  equally  important  payoff  from  serving  the 
information  needs  of  the  technician  is  that  there  will  be  an  incentive  to 
provide  accurate  input  data  to  such  a  system.  There  is  no  such  incentive  now; 
filling  out  the  necessary  3-M  forms  is  only  additional  work  with  no  benefit 
for  the  technician.  There  is  no  need  to  elaborate  on  the  repercussions  in 
terms  of  data  inaccuracies  in  the  present  3-M  data  base. 


123NAVMAT,  Integrated  Functional  Description  for  SNAP  II.  30  March  1981. 

12*LT  p.  g.  Smith,  USN,  SNAP  II:  Shipboard  Micro  Computer  Applications 
in  Personnel,  Administration,  and  Training  -  A  User's  Perspective  (M.S. 
Thesis,  Monterey,  California:  Naval  Postgraduate  School,  December  1981). 

123LCDR  John  A.  Dollard,  USN,  Computer-Based  Shipboard  Personnel 
Readiness  and  Training  Management  System:  Subsystem  Specifications.  NPRDC  SR 
82-5  (San  Diego:  Navy  Personnel  Research  and  Development  Center,  October 
1981) . 
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TECHNICAL  MANUALS 

An  assessment  of  current  problems  in  TM  acquisition  and  support  and  an 
outline  of  needed  improvements  follow.  (THs,  by  our  terminology,  include 
hardcopy  maintenance  job  aids;  electronic  maintenance  aids  are  discussed 
separately. ) 

Management  control  of  TM  acquisition  and  support  is  exercised  by 
centralized  offices  within  NAVSEA  for  all  nonaviation  TMs,  and  within  NAVAIR 
for  all  aviation  TMs.  Each  is  supported  by  a  centralized  field  support 
activity  responsible  for  post-acquisition  management,  including  configuration 
management  and  distribution.  These  activities  are  the  Naval  Sea  Data  Support 
Activity  and  the  Naval  Air  Technical  Services  Facility  (NATSF),  respectively. 
Both  use  automated  management  information  systems;  the  Ships  Technical 
Publication  System  (installed  in  1981),  and  the  NATSF  Information  Management 
System  (NIMS,  tested  in  1983  and  being  installed  now).  In  view  of  the 
gigantic  inventory  of  THs,  these  systems  are  crucial  for  post-acquisition 
management;  e.g.,  tracing  the  impacts  of  alterations,  tracking  the  status  of 
engineering  changes  and  deficiency  reports  received  from  the  Fleet,  and 
establishing  TM  update  priorities.  The  recent  implementation  of  these  systems 
may  alleviate  some  of  the  problems  experienced  in  the  past,  such  as  the 
following: 

a.  Close  to  one  hundred  military  specifications  are  used  in  TM 
acquisition.  While  the  Navy  has  implemented  a  modular  specification  system 
tailoring  TM  contract  specifications  to  the  associated  equipment,  lack  of 
standardization  (format,  structure,  sequence,  illustrations,  terminology, 
presentation  technique)  remains  a  problem,  influencing  TM  acceptance  and  use 
by  technicians. 

b.  Over  50  percent  of  new  THs  are  not  verified  prior  to  distribution  to 
the  Fleet.126  Such  "preliminary"  TMs  cause  confusion  among  the  technicians 

as  to  their  validity  and  deficiency  reporting  requirements. 

c.  Deficiency  reporting  by  TM  users  is  incomplete:  the  process  is 
focused  on  reporting  errors,  not  voids;  over  20  percent  of  technicians  do  not 
report  TM  deficiencies  because  the  "system"  is  unresponsive.127 

d.  Serious  delays  are  encountered  in  updating  TMs  as  a  result  of 
limited  budgets.  Approved  changes  are  frequently  deferred  two  to  four  years; 


1260ff  ice  of  the  Inspector  General,  Department  of  Defense.  Audit 
Report:  Review  of  Technical  Manuals  (Report  Number  83-128),  19  May  1983. 

127The  statistics  are  based  on  a  survey  of  177  electronics  maintenance 
technicians,  as  documented  in:  NTIPP  Fleet  Survey  of  Technical  Manual  Users 
(Bethesda,  Maryland:  David  W.  Taylor  Naval  Ship  Research  and  Development 
Center,  January  1978). 
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many  changes  never  make  it  before  the  associated  equipment  is  replaced  by  a 
major  upgrade  (new  model)  or  is  retired  from  service  (see  footnote  126). 

e.  THs  are  deemed  inadequate  by  technicians:  12(5 

(1)  Information  is  incomplete.  Insufficient  coverage  is  reported 
by  30  percent  for  equipment  description,  50  percent  for  theory  of  operations, 
44  percent  for  troubleshooting  procedures,  80  percent  for  test  equipment  setup 
and  use. 


(2)  Information  is  inaccurate.  30  percent  believe  theory  of 
operations  is  inaccurate;  32  percent  of  technicians  (51  percent  of  operators) 
report  their  TMs  are  out  of  date. 

(3)  Information  is  incomprehensible.  23  percent  do  not  understand 
theory  of  operations;  90  percent  of  those  using  the  Functionally  Oriented 
Maintenance  Manual  (FOMM)  format  for  troubleshooting  do  not  understand  it;  25 
percent  do  not  understand  the  illustrations/graphics  in  their  TMs;  30  percent 
state  they  need  more  illustrations. 

The  consequences  of  inadequate  TMs  are  significant.  The  average  novice 
becomes  very  dependent  on  the  supervisor  or  more  experienced  peers  for 
developing  skills  through  "show  and  tell";  the  novice  cannot  get  it  from  the 
manual.  As  long  as  those  people  are  available  to  "hold  his  hand,"  there  is  no 
problem.  When  they  are  not  available  or  have  acquired  poor  maintenance  habits 
themselves  (as  a  result  of  inadequate  school  training  and  negative 
reinforcement),  there  is  a  problem.  Furthermore,  the  overall  result  is  a  poor 
return  on  investment  in  TMs:  novices  cannot  use  them,  journeyman  technicians 
do  not  use  them  (they  develop  their  own  pocket  guides  or  get  them  from 
contractors).  This,  in  turn,  contributes  to  a  common  perception  that  spending 
money  on  improving  the  quality  of  TMs  is  a  waste:  TM  management  line  items  in 
the  O&M  budget  invariably  suffer  from  a  disproportionate  share  of  budget  cuts. 

The  above  problems  experienced  with  TMs  in  the  past  were  not  peculiar  to 
the  Navy.  DoD-wide  problems  with  TM  management  have  been  noted  in  several 
Inspector  General  and  GAO  reports.  This  resulted  ultimately  in  a  revision  of 
DoD  policy  (DoD  Instruction  4151.9,  "DoD  Technical  Manual  Program  Management, 
February  1982),  tightening  the  requirements  for  TMs  in  many  respects 
including:  technical  accuracy  and  adequacy,  completeness,  comprehensibility, 
validation  and  verification,  and  configuration  control  with  timely  issuance  of 
TM  changes  or  revisions.  It  also  reemphasized  TM  specifications  and  standards 
and  the  Army's  continued  lead  role  in  standardization;  and  assigned  the  Navy 
to  take  the  lead  in  ".  .  .  R&D  of  advanced  technology  applicable  to  the 
development,  presentation,  use,  storage,  and  retrieval  of  TM  information." 


i?8Ibid. 
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a.  Formulating  the  Mess.  Capturing  the  essential  systemic  properties 
of  the  mess.  Section  II  of  this  report  (especially  the  "Summary  of  Case 
Studies")  provides  this  information.  It  should  be  reviewed  and  discussed  as  a 
starting  point  in  the  process. 

b.  Ends  Planning.  Defining  the  objectives  and  goals  to  be  pursued.  If 
agreement  is  reached  that  the  objective  is  "proficient  maintenance  performance 
enabling  attainment  of  operational  availability  goals  of  installed  combat 
systems  by  authorized  personnel  assets,  while  minimizing  dependency  on  Fleet 
support  resources  and  excess  spares,"  then  the  gaps  to  be  filled  by  the  plan 
are  identified  by  the  information  in  Section  II. 

c.  Means  Planning.  Selecting  the  ways  of  filling  the  gaps.  They  can 
take  the  form  of  policies,  research  projects,  procedures,  or  management 
action.  The  crucial  issues  are  discussed  in  Section  IV.  Some  will  require 
further  research;  some  do  not  need  further  research  —  just  deciding  and 
agreeing  upon  a  course  of  action. 

d.  Resource  Planning.  Determining  when,  how  much,  and  what  type  of 
resource  (people,  equipment,  funds,  information,  knowledge)  will  be  required 
by  the  means  selected.  If  the  resources  required  cannot  be  met,  the 
previously  formulated  means  must  be  revised  and  the  cycle  repeated. 

e.  Design  of  Implementation.  Deciding  who  does  what,  where,  and  when, 
and  how  implementation  is  to  be  monitored  and  modified  when  necessary. 

This  approach  appears  obvious,  but  there  are  no  indications  that  the  Navy 
has  ever  addressed  the  maintenance  performance  problem  in  a  systematic  way. 

This  may  explain  why  previous  attempts  to  deal  with  the  problem  have  been 
largely  unsuccessful.  The  problem,  however,  is  not  insurmountable.  There  is 
no  doubt  that  the  Navy  can  eliminate  the  problem,  given  sufficient  management 
attention  and  priority.  Now  is  the  time  to  do  so,  because  the  problem  has 
reached  serious  proportions,  endangering  the  Navy's  sustainability. 

Such  an  approach  may  be  useful  in  developing  and  executing  a 
comprehensive  Maintenance  Improvement  Plan  to  meet  the  following  needs: 

a.  Provide  top-level  sponsorship  and  coordination  of  the  two  major  RSJ) 
programs  currently  in  test  and  evaluation:  EPICS  and  NTIPS.  Both  are  crucial 
for  eliminating  many  factors  which  adversely  impact  maintenance.  While  both 
are  sound  in  concept  (multilevel,  distributed  training  and  technical  data 
automation,  respectively),  tests  will  show  whether  the  "frills"  which  have 
been  added  over  time  are  necessary  and  effective.  Implementation  of  these 
systems  will  involve  radical  changes  in  the  Navy's  traditional  way  of  doing 
business  with  far-flung  implications  in  many  different  functional  areas.  For 
this  reason,  implementation  is  bound  to  fail  without  aggressive,  top-level 
sponsorship.  Because  the  two  systems  evolved  in  isolation  of  one  another,  but 
have  major  overlaps,  they  are  not  coordinated.  This  must  be  corrected  in  the 
implementation  planning  for  each.  -  -  i 


110 


NAVTRAEQUIPCEN  MDA903-81-C-0166-1 


SECTION  V 
RECOMMENDATIONS 


The  intent  of  this  study  was  to  identify  R&D  needs  in  the  areas  of 
maintenance  training  and  aiding  and  to  recononend  specific  R&D  projects  with  a 
high  payoff  in  terms  of  potential  maintenance  performance  improvements.  This 
section  first  touches  upon  the  need  for  a  much  broader,  systematic  plan  to 
effect  maintenance  improvements,  and  then  focuses  upon  the  specific  R&D  needs 
within  the  scope  of  that  plan.  Candidate  R&D  projects  are  listed  in  Appendix 
A. 

MAINTENANCE  IMPROVEMENT  PLAN 

R&D  programs  in  training  and  aiding  technology  must  be  coordinated  within 
the  much  broader  context  of  a  systematic  plan  in  order  to  reap  the  potential 
benefits  from  R&D  investments.  Better  training  equipment  and/or  job  aids  will 
pay  off  in  improved  maintenance  performance  only  if  the  adverse  maintenance 
impacts  of  other  factors  are  eliminated.  Some  of  those  factors  (e.g.,  weapon 
system  design,  maintenance  policies,  training  philosophy,  and  instructional 
system  development)  have  an  overwhelming  impact  on  maintenance  performance;  if 
left  alone,  they  would  negate  the  potential  improvements  offered  by  more 
effective  training  equipment  or  job  aids.  Other  factors  (e.g.,  personnel 
management,  supply  support)  may  carry  less  weight,  but  still  would  limit  the 
potential  payoffs  from  better  training  equipment  or  job  aids. 

The  Navy  currently  lacks  a  systematic  plan  for  focusing  R&D  effort  or 
management  action  on  the  factors  contributing  to  maintenance  shortfalls. 
Development  of  such  a  plan  requires  a  carefully  structured  process  because  the 
problem  at  hand  is  complex  and  systemic.  An  effective  approach,  under  these 
circumstances,  is  provided  by  Ackoff's  "design  approach  to  mess 
management."148  Ackoff  defines  this  as  an  approach  to  dissolve  a  problem 
(changing  the  characteristics  of  the  larger  system  containing  the  problem  so 
as  to  remove  the  problem),  in  contrast  to  the  clinical  approach  to  resolve  a 
problem  (selecting  a  course  of  action  yielding  a  satisfying  outcome),  or  the 
research  approach  to  solve  a  problem  (obtain  the  best  possible,  optimizing 
outcome).  The  design  approach  synthesizes  the  clinical  and  research 
approaches  but  focuses  on  managing  a  mess  (defined  as  a  dynamic  system  of 
problems),  not  problem  solving.  It  involves  a  structured  planning  process 
carried  out  participatively  by  all  principals  responsible  for,  or  holding  a 
stake  in  dissolving  the  mess.  They  are  organized  into  small  planning  teams, 
managed  to  assure  coordination  of  effort,  and  supported  by  planners  and 
researchers,  as  needed.  Ackoff  defines  the  following  five  phases  for  this 
process,  which  are  paraphrased  for  the  purposes  of  the  present  case: 


l48Russell  A.  Ackoff,  "The  Art  and  Science  of  Mess  Management," 
Interfaces ,  Vol.  II,  No.  1  (February  1981). 
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mission  availability  (availability  when  needed  in  class  hours),  not  average 
annual  availability.  Samples  of  current  contractual  requirements  applicable 
to  nonaviation  training  equipment  suggest  that  the  Navy  needs  to  tighten  the 
requirements . 

These  issues  will  be  addressed  again  in  the  R&D  recommendations  in  the 
next  section. 

SUMMARY 

In  summary,  most  of  the  key  factors  responsible  for  today's  poor 
maintenance  performance  in  the  Navy  are  well-known  and  have  been  studied  and 
analyzed  in  the  past.  The  fact  that  the  shortcomings  identified  in  this 
section  have  so  far  not  caused  much  more  degradation  in  maintenance  than  our 
assessment  indicates  is  a  credit  to  the  Fleet  and  its  ability  to  somehow 
compensate  or  work  around  these  shortcomings.  Nevertheless,  as  discussed  in 
Section  II,  the  Fleet  feels  it  has  reached  the  limit;  and  as  discussed  in 
Section  III,  the  situation  is  bound  to  get  worse  in  the  future,  unless  drastic 
improvements  are  made.  To  a  large  extent,  the  answers  as  to  what  changes  are 
necessary  are  known.  What  appears  to  be  lacking  is  an  aggressive  plan  of 
action  to  develop  and  implement  the  necessary  changes.  The  R&D  efforts  to 
move  the  Navy  toward  improvements  on  the  material  side  (design  for  testability 
and  integrated  diagnostics)  appear  to  be  well  orchestrated.  Similar 
management  action  is  necessary  on  the  personnel  and  training  side  if  the  Navy 
is  to  solve  successfully  the  problem  of  inadequate  maintenance.  The  next 
section  provides  an  outline  of  a  systematic  R&D  plan  in  support  of  such  action. 
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discussed  at  the  Fourth  Interservice/Industry  Training  Equipment  Conference 
(Orlando,  November  1982);  and  is  planned  as  an  interservice  R&D  effort. 

g.  User  Involvement.  The  Fleet  Project  Team  approach  has  traditionally 
been  used  to  involve  the  ultimate  users  (instructors)  in  developing  the 
specifications  for  new  training  equipment.  Our  survey  showed  that  gaining 
instructor  acceptance  is  possibly  the  most  crucial  factor  determining 
utilization  and  effectiveness  of  training  equipment.  Acceptance  is  influenced 
by  "perceived  validity,"  a  factor  which  is  difficult  to  quantify,  not  always 
related  to  functional  fidelity,  and  essentially  based  on  instructor 
prejudice.  Other  factors  influencing  acceptance  are:  (1)  responsibility  for 
trainer  maintenance,  (2)  method  of  trainer  introduction,  (3)  ease  of  use  by 
the  instructor,  and  (4)  extent  of  curriculum  support.  All  studies  on  the 
issue  of  user  acceptance  agree  on  the  necessity  of  user  involvement  in 
developing  trainer  design  specifications.  At  the  same  time,  experience  has 
shown  that  instructors  should  not  be  allowed  to  exercise  too  much  control.147 

h.  Fidelity.  "Physical  fidelity"  has,  in  the  past,  often  been  confused 
with  "functional  fidelity"  and/or  "psychological  fidelity."  There  is  an 
increasing  body  of  evidence  that  too  much  emphasis  on  physical  fidelity  may 
limit  training-effectiveness,  especially  in  the  area  of  cognitive  skills. 
Functional  fidelity  has  been  shown  to  be  more  important  for  training¬ 
effectiveness,  while  psychological  fidelity  has  been  shown  to  influence  both 
instructor  and  trainee  acceptance.  Additional  work  is  required  to  develop 
measures  or  specifications  for  these  important  attributes  and  to  assess  more 
precisely  how  each  relates  to  transfer-of-training  as  a  function  of  training 
objectives . 

i.  Logistic  Support.  Poor  support  of  training  equipment  has  influenced 
use:  instructors  tend  to  use  the  equipment  for  demonstrations  vice  student 
hands-on  practice,  and  when  the  equipment  is  "down,"  it  may  remain  unavailable 
for  a  long  time.  The  Navy  is  in  the  process  of  transitioning  to  contractor 
support  of  all  training  equipment.  If  properly  funded,  this  transition  should 
tend  to  increase  the  use  of  training  equipment  as  planned.  Because  past 
support  costs  are  a  poor  indicator  of  future  support  needs  (including 
configuration  changes),  the  Navy  needs  to  develop  better  estimating  procedures 
to  ensure  that  budget  requests  will  reflect  true  support  requirements. 
Furthermore,  contractual  support  requirements  must  be  specified  in  terms  of 


147The  troubleshooting  training  courseware  for  the  A-7E  HUD  Test  Set 
simulator  was  specified  to  a  large  extent  by  one  instructor.  By  the  time  the 
simulator  was  delivered,  a  different  instructor  gave  the  course;  the  new 
instructor  objected  to  the  courseware  as  delivered.  This  was  one  of  the 
lessons  learned  in  the  Simulated  Avionics  Maintenance  Trainers  program  and 
ultimately  resulted  in  a  new  procurement  specification  for  front-end 
analysis:  General  Specification  for  Procurement  of  Maintenance  Training 
Simulator  Functional  Requirements,  NAVTRAEQUIPCEN  N-712-357  (Orlando, 

Florida:  Naval  Training  Equipment  Center,  January  1983). 
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c.  Cost-Effectiveness .  Because  training-effectiveness  is  not  properly 
assessed,  cost-effectiveness  is  not  either.  To  establish  the  case  for 
simulation,  hard  data  must  be  developed.  Rigorous  cost-effectiveness  data  are 
still  lacking  today.146 

d.  Media  Mix.  Bits  and  pieces  of  research  data  are  available  for 
relating  training  equipment  design  features  and  training  objectives. 
Nevertheless,  by  lack  of  a  coordinated  research  effort,  the  knowledge  for 
determining  the  optimal  mix  of  devices  is  not  there.  Previous  authors  (see 
footnote  95)  have  articulated  the  need  for  research  to  determine,  the  unique 
contribution  each  type  of  training  device  can  make,  the  ways  they  can 
complement  one  another,  the  training  methods  which  amplify  the  advantages  of 
each,  the  types  of  training  objectives  most  conducive  to  each,  and  the  extent 
of  hands-on  practice  required  (in  a  "functional  context"  approach  to  learning) 
in  order  to  minimize  forgetting  knowledge  learned.  Research  in  this  area  has 
been  limited. 

e.  Acquisition  Guide.  In  the  present  situation,  the  prime  equipment 
program  manager  has  every  reason  to  decide  on  weapon  system-peculiar  training 
equipment.  This  may  not  always  be  the  best  use  of  limited  training  equipment 
funds  for  the  Navy  as  a  whole.  In  order  to  move  toward  more  cost-effective 
decisions,  the  cost-benefits  of  alternative  options  must  be  articulated  in  an 
acquisition  guideline  which  currently  does  not  exist.  The  key  questions  to  be 
addressed  by  such  a  guide  are:  (1)  Is  new  training  equipment  required?  (2) 
What  type  is  most  cost-effective,  actual  or  simulated?  (3)  If  simulated,  what 
choice  is  best:  general  purpose,  generalized,  or  equipment-replica?  (4)  What 
training  features  are  important? 

f.  Standardization.  Most  equipments  designed  for  use  in  training  have 
certain  features  in  common;  i.e.,  there  is  an  instructor  station,  one  or  more 
student  stations,  a  general-purpose  computer  with  courseware,  and  bits  and 
pieces  of  operational  equipment.  It  should  be  possible  to  reduce  training 
equipment  costs  by  several  orders  of  magnitude  by  standardizing  (form,  fit, 
function)  the  components  of  such  training  systems.  The  modular  concept  for 
training  simulators  is  a  recent  SIMSPO  (the  Air  Force's  Simulator  Systems 
Program  Office)  initiative,  addressing  both  hardware  and  software;  was 


146The  most  authoritative  account  of  the  limitations  of  available  cost- 
effecti  veness  studies  of  maintenance  training  simulators  is  provided  by:  J. 
Orlansky  and  J.  String,  Cost-Effectiveness  of  Maintenance  Simulators  for 
Military  Training.  IDA  Paper  P-1568  (Arlington,  Virginia:  Institute  for 
Defense  Analysis,  August  1981).  One  of  the  few  serious  cost-effectiveness 
evaluations  conducted  to  date  is  described  in:  Louis  F.  Cicchinelli,  Kenneth 


Actual  Equipment.  AFHRL  TR-82-30  (San  Antonio,  Texas:  Air  Force  Human 
Resources  Laboratory,  Brooks  Air  Force  Base,  December  1982).  But,  as  its 
authors  indicate,  the  latter  study  still  ignores  training  adequacy. 
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but  exhibiting  poor  training  effectiveness  and/or  limited  utilization.143 
It  is  unnecessary  to  elaborate  on  these  issues.  However,  what  is  inhibiting 
the  Navy  from  making  much  progress  is  the  lack  of  institutional  knowledge 
about  training-effectiveness  and  cost-effectiveness  of  training  devices  or 
simulators,  and  the  associated  equipment  design  factors.  One  reason  is  a  void 
in  knowledge  in  spite  of  30  years  of  research  on  training  equipment.  Another 
reason  is  the  lack  of  follow-up  and  documentation  of  lessons  learned  with  each 
training  device  that  is  installed. 

Some  of  the  critical  issues  that  must  be  answered  through  additional  RAD 
include  the  following: 

a.  Job-Task  Analysis.  Current  procedures  for  job-task  analysis  are 
known  to  be  inadequate  for  identifying  the  functional  requirements  for  a 
training  device.144  A  better  procedure  must  be  developed  for  this  purpose. 

b.  Evaluation.  The  effectiveness  of  a  new  training  device  continues  to 
be  evaluated  by  comparing  end-of-course  performance  tests  of  trainees  trained 
with  the  new  device  and  those  trained  conventionally  on  the  actual  equipment. 
The  lack  of  validity  of  this  type  of  evaluation  has  been  addressed  by  numerous 
authors  in  the  past.143  The  point  is  that  some  training  objectives  are 
better  served  through  simulators;  some  require  nothing  more  than  a  wall  chart; 
and  some  may  be  trained  best  on  actual  equipment  trainers.  Another  point  is 
that  end-of-course  performance  testing  has  limited  validity  and  should  be 
supplemented  with  operational  field/fleet  performance  testing. 


143Naval  Material  Command,  Training  Equipment  Management  Study 
(December  1980). 

144See,  for  example,  the  following  references:  Villiam  E.  Montague,  Is 
Simulation  Fidelity  the  Question?.  NPRDC  TN  82-13  (San  Diego:  Navy  Personnel 
Research  and  Development  Center,  April  1982);  Nicholas  A.  Siecko,  "Front  End 
Analysis  Lessons  Learned  from  Maintenance  Simulator  Design,"  Proceedings  of 
Conference  on  Front  End  Analysis  for  Simulator  and  Device  Based  Training  (July 


1961);  and  Bertram  V.  Cream,  F.  Thomas  Eggemeier,  and  Gary  A.  Klein,  "A 
Strategy  for  the  Development  of  Training  Devices,"  Human  Factors  (1977).  The 
need  for  more  basic  research  in  this  area  was  first  pointed  out  by  Glenn  L. 
Bryan  and  James  J.  Regan,  "Training  System  Design"  (Chapter  13  in  Harold  P. 
Vancott  and  Robert  G.  Kinkade  (eds.).  Human  Engineering  Guide  to  Equipment 
Design  (Washington,  D.C.:  U.S.  Government  Printing  Office,  1972). 

143See,  for  example,  the  following  references:  Melvin  D.  Montemerlo, 
"Maintenance  Training  Devices:  Data  and  Directions,"  Proceedings  of  the  1977 
Summer  Computer  Simulation  Conference  (Chicago:  July  1977);  and  C.  Dennis 
Fink  and  Edgar  L.  Shriver,  Simulators  for  Maintenance  Training:  Some  Issues. 
Problems,  and  Areas  for  Future  Research,  AFHRL  TR-78-27  (Denver,  Colorado: 


Air  Force  Human  Resources  Laboratory,  Lowry  Air  Force  Base,  July  1978). 
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replace  TNs;  the  latter  will  always  be  needed  as  a  reference  source  —  paper, 
if  properly  protected,  is  still  the  most  reliable  information  storage  medium 
invented  by  man.  Instead,  cost-effectiveness  of  aids  should  be  judged  on  the 
basis  of  reduction  of  OJT  duration  to  achieve  proficiency. 

The  PEAK  (Personal  Electronic  Aid  for  Maintenance)  R&D  project  is  clearly 
headed  in  the  right  direction.  Additional  R&D  effort  is  needed,  it  appears, 
in  developing  a  better  understanding  of  what  differentiates  an  expert  from  a 
non-expert  troubleshooter,  and  what  tutorial  software  is  needed  to  transform 
the  latter  into  the  former.  Artificial  intelligence  has  a  lot  to  offer  in 
this  area,  and  will  be  addressed  in  the  R&D  recommendations  in  Section  V. 

TRAINING  EQUIPMENT 

The  Navy's  investment  in  training  equipment  is  large  ($8  billion 
cumulative  procurement  cost  of  equipment  as  of  1982);  most  (85  percent)  of  the 
equipment  is  unmodified,  operational  equipment. On  a  procurement  cost 
basis,  most  is  used  for  operator  training;  only  five  percent  of  the  training 
device  investment  to  date  was  spent  on  devices  used  for  maintenance  skill 
training. But  in  recent  years,  the  trend  is  shifting  toward  increased 
investment  in  training  devices;  over  50  percent  of  the  device  inventory  in 
current  use  was  procured  in  the  last  five  years,  and  the  cumulative 
procurement  cost  of  installed  devices  is  growing  by  15  percent  a  year.*42 
While  the  Navy  is  still  spending  three  times  more  on  operational  equipment 
($300  million  annually)  than  on  devices  ($100  million  annually),  this  ratio  is 
projected  to  shift  further  in  favor  of  the  latter.  This  trend  is  attributed 
to  the  steep  cost  increase  of  operational  equipment. 

Previous  Navy  studies  have  identified  the  need  for  substantial  changes  or 
improvements  in  organization,  management  procedures  and  practices  to  counter 
the  proliferation  of  weapon  system-specific  training  equipment  at  huge  cost 


140Naval  Material  Command,  Training  Equipment  Management  Study. 

December  1980. 

141Naval  Training  Equipment  Center,  Financial  Accounting  and  Inventory 
Record  NAVTRAEQUIPCEN  Cognizance  Symbol  2"0".  Special  Report  as  of  January  19, 
1983.  (The  Navy  classifies  training  equipment  into  two  categories:  (1) 
technical  training  equipment  which  is  operational  equipment  under  cognizance 
of  a  Systems  Command,  and  (2)  training  devices  which  are  any  devices  or  aids 
used  for  training  purposes  and  under  the  cognizance  of  the  NAVTRAEQUIPCEN). 

142Naval  Training  Equipment  Center,  Cognizance  Symbol  2"0"  Training 
Equipment-Device  Accountability  Program:  Management  Summary.  30  September 
1981. 
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troubleshooting  performance  has  received  only  very  limited  attention  by 
training  and  aiding  professionals. 

f.  Because  such  aids  are  fully  prescriptive,  they  are  not  well  accepted 
by  users.  And  if  forced  to  use  them,  userB  will  try  to  take  shortcuts  vhich 
invariably  lead  them  astray  in  the  process. 

g.  For  the  same  reasons,  individuals  who  are  forced  to  use  such  aids  do 
not  learn  anything  from  using  the  aid.  This  absence  of  skill  development  on 
the  job  is  probably  the  most  heinous  and  self-defeating  feature  of  the  whole 
concept . 

In  sum,  the  concept  of  fully  proceduralized  troubleshooting  aids  was 
naive  and  did  not  consider  the  human  implications;  in  practice,  such  aids  can 
be  applied  to  only  a  fraction  of  system  failures  and  their  accuracy  is  subject 
to  the  quality  of  configuration  management,  which  is  often  poor.  Yet  the 
concept  relies  on  totally  accurate  procedures.  Verification  of  these 
procedures  is  very  hard  in  the  absence  of  a  failure  simulation  or  testability 
analysis  model.  For  example,  there  are  many  instances  in  which  a  "top-down" 
logic  is  applied,  whereby  the  procedure  tells  the  technician  to  replace  a 
failed  module  without  first  ascertaining  whether  failures  or  out-of-tolerance 
conditions  are  present  elsewhere  which  may  have  caused  the  first  module  to 
fail.  Replacement  of  the  latter  does  not  correct  the  cause  and  will  result  in 
the  same  failure  symptoms  again.  Nevertheless,  a  maintenance  demonstration  or 
operational  test  may  not  surface  such  deficiencies  in  troubleshooting 
procedures. 

CRITICAL  ISSUES.  Among  the  numerous  requirements  which  an  electronic  job  aid 
must  meet  to  be  operationally  successful  (small,  portable,  fast,  reliable, 
etc.),  the  single  most  important  requirement  is  flexibility.  The  software 
must  be  designed  such  that  the  user  of  the  aid  gets  the  information  needed,  no 
more,  no  less.  The  experienced  technician  needs  a  diagram  and  tolerance 
values  of  test  points  (he  might,  of  course,  prefer  to  have  this  in  paper 
format,  eliminating  dependency  on  the  aid);  a  less  experienced  technican  may 
want  to  use  the  aid  interactively,  having  it  dictate  what  measures  to  make  and 
feeding  it  his  observations;  a  novice  needs  additional  data  on  the  system, 
nomenclature,  locations,  etc. 

Traditionally,  aids  (paper  or  electronic  medium)  have  been  justified  on 
the  basis  of  obtaining  some  productivity  from  inexperienced  personnel.  And 
this  attitude  has  influenced  very  much  the  naive  notion  of  fully 
proceduralized  troubleshooting  aids.  The  most  important  objective  of  an  aid 
should  be  to  expedite  skill  development  on  the  job  through  emphasizing 
learning  while  using  the  aid.  The  art  of  designing  an  effective  aid  lies  in 
providing  the  apprentice-technician  sufficient  challenge  and  learning  from 
task  performance  so  that  a  level  of  proficiency  is  reached  whereby  the  aid  is 
no  longer  needed,  except  for  reference  data.  This  transitioning  is  key  to  the 
acceptance,  utilization,  and  effectiveness  of  a  troubleshooting  aid  in  the 
real  world.  Thus,  aids  should  not  be  justified  on  the  basis  of  life-cycle 
cost  comparisons  with  TNs.  Such  comparisons  are  irrelevant.  Aids  will  never 
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which  he  must  deduce  what  to  do;  and  hybrid  aids  which  combine  the 
characteristics  of  prescriptive  and  deductive  aids  in  various  degrees. 
Physically,  prescriptive  aids  provide  sequences  of  steps  or  instructions, 
deductive  aids  provide  graphics  or  diagrt^is. 

Thus,  aids  come  in  many  different  forms.  The  increasing  power  of 
microcomputers  has  made  aiding  technology  practical  and  cost-effective.  But 
operational  use  of  such  aids  by  the  Services  is  very  limited  at  present.  Some 
aids  have  been  tested  or  demonstrated;  most  planned  aiding  application  are 
still  in  R&D. 

LESSONS  LEARNED.  One  of  the  reasons  for  caution  in  the  development  and 
implementation  of  electronic  job  aids  is  that  the  experience  with  aids  in 
paper  format  has  not  lived  up  to  the  claims  or  promises  made  by  their 
proponents.  Especially  in  the  area  of  fully  proceduralized  troubleshooting 
aids,  the  experience  has  been  dismal.  The  Navy  has  wisely  preferred  a 
deductive  format  (FOMM,  HIL-M-24100)  for  aids  supporting  complex  systems,  but 
due  to  higher  costs,  such  aids  are  still  a  minority.  To  avoid  making  the  same 
mistakes  with  electronic  job  aids,  it  may  be  useful  to  recount  why  fully 
proceduralized  troubleshooting  aids  in  paper  format  failed.  The  following 
factors  appear  to  be  responsible: 

a.  Combat  systems  are  complex;  the  acquisition  process  does  not  allow  a 
complete  failure  modes  and  effects  analysis  to  document  all  failures  (symptoms 
and  causes)  which  may  occur  in  the  operational  environment.  The  aids, 
therefore,  cannot  be  complete  unless,  and  until,  an  automated  testability 
analysis  has  become  institutionalized  in  the  acquisition  process. 

b.  The  operational  environment  is  demanding;  using  a  fully 
proceduralized  aid  requires  excessive  time  and  may  not  be  successful,  but  the 
single  operational  criterion  is  minimizing  system  downtime.  Such  aids  are, 
therefore,  often  impractical  at  the  organizational  maintenance  level. 

c.  Engineering  changes  are  frequent  in  combat  systems;  hundreds  of 
engineering  changes  typically  occur  in  the  early  operational  years  of  a  new 
combat  system  and  by  the  time  this  stabilizes,  a  new  model  is  typically 
fielded.  The  problem  of  keeping  aids  up  to  date  is  exacerbated  by  the  volume 
of  changes  necessitated  by  fully  proceduralized  aids.  let,  the  utility  of 
such  aids  is  totally  dependent  upon  the  accuracy  and  completeness  of  the 
information. 

d.  Failure  modes  and  their  relative  frequencies  are  influenced  by 
platform  installation.  That  is,  the  same  equipment  behaves  differently  on 
different  platforms.  To  be  fully  effective,  proceduralized  aids  should  differ 
among  different  platforms,  but  technical  data,  including  aids,  are  equipment- 
specific,  not  platform-specific. 

e.  Failure  detection  and  isolation  is  very  much  a  probabilistic,  not  a 
deterministic  process.  Test  points  may  exhibit  marginal  values  which  are 
insufficient  to  make  a  decision  with  certainty.  The  impact  of  uncertainty  on 
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There  is  only  one  way  to  provide  cost-effective  supply  support  and  that 
is  through  relating  stockage  requirements  to  weapon  system  availability, 
considering  item  costs,  mission  scenarios,  item-to-mission  essentiality, 
repair  cycle  requirements,  and  other  parameters.  So  far,  the  Navy  has  not 
been  successful  in  developing  an  appropriate  model  for  this  purpose,  although 
sparing-to-availability  models  have  been  used  in  computing  initial 
provisioning  budgets  for  selected  systems  (e.g.,  AEGIS,  SURTASS,  and  LAMPS  Mk 
3).  The  Ship  Support  Improvement  Project  has  been  working  on  an  "A0 
Allocation  Model,"  but  it  is  still  in  the  early  stage  of  development.  The 
need  to  replace  traditional  commodity  orientation  of  the  supply  system  by  a 
weapon  system  orientation  will  become  more  urgent  with  the  increasing 
complexity  (parts  counts  and  costs)  and  parts  reliability  of  new  weapon 
systems.  The  technology  involved  is  well  known.139  Its  implementation 
requires  more  detailed  configuration  and  mission-essentiality  data  than 
provided  in  current  equipment  data  files  supporting  COSAL  and  AVCAL 
computations.  The  Navy  needs  to  proceed  in  this  direction,  starting  with 
aircraft  material  and  shipboard  combat  systems. 

ELECTRONIC  JOB  AIDS 

Electronic  maintenance  aids  are  loosely  defined  as  any  stand-alone 
devices  (including  associated  software,  data  base,  and  input/output  media) 
which  aid  the  technician  in  performing  maintenance  tasks  or  improving 
maintenance  performance.  "Stand-alone"  means  it  is  not  built  into  the  prime 
equipment  (i.e.,  BIT  equipment),  nor  is  it  hooked  up  to  the  prime  equipment 
(i.e.,  manual  or  ATE).  It  "aids"  the  technician  by  presenting  information 
beyond  that  otherwise  obtainable  from  an  automated  maintenance  data  system 
(i.e.,  maintenance  log  or  history).  The  information  presented  depends  upon 
the  maintenance  task  involved  and  the  sophistication  of  the  software.  The 
simplest  type  of  aid  duplicates  the  information  presented  in  the  TH.  Such  an 
aid,  commonly  referred  to  as  an  "electronic  page  turner,"  improves 
retrievability  of  the  information  (compared  to  the  paper  medium)  and  reduces 
the  probability  of  human  errors  in  lengthy  troubleshooting  procedures.  If 
designed  for  interactive  use  by  the  technician,  the  aid  will  "step”  the 
technician  through  the  proper  procedure.  A  more  sophisticated  aid  may  be 
designed  to  present  heuristic  procedures  or  strategies  which  have  been 
developed  by  experts  and  are  continuously  adjusted  based  on  failure 
experience.  The  most  efficient  aid  for  troubleshooting  is  based  on  an 
analytical  model  of  the  prime  equipment,  incorporating  all  failure  symptom- 
cause  relationships  and  identifying  the  minimum  steps  (or,  the  sequence  of  the 
easiest  steps)  required  to  isolate  a  failure.  Another  way  of  classifying  aids 
is  by  type  of  information  presentation:  prescriptive  aids  which  tell  the 
technician  exactly  what  to  do;'  deductive  aids  showing  the  technician  the 
maintenance  dependency  chains  (functional  symptom-cause  relationships)  from 


139T.  J.  O'Malley,  The  Aircraft  Availability  Model:  Conceptual 
Framework  and  Mathematics.  LMI  Task  AF201  (Washington,  D.C.:  Logistics 
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(2)  It  ignores  item  costs,  thereby  inhibiting  accommodation  of  cost 
constraints  other  than  through  awkward  iterative  procedures. 

(3)  It  assumes  a  single  Poisson  distribution  for  the  probability  of 
item  demands  with  no  empirical  justification  found  for  this  assumption. 

d.  The  recent  modification  in  the  FLSIP  model136  (lowering  the 
threshold  on  item  demand  rate  from  0.25  to  0.10  demands  per  year  to  qualify 
for  allowance  as  an  insurance  item  and  increasing  the  allowance  for  insurance 
items  from  1  to  2  for  item  demand  rates  falling  between  2.0  and  4.0  demands 
per  year  —  both  changes  for  mission-essential  items  only)  will  reduce  the 
probability  of  mission-degrading  stock-outs,  but  a  similar  improvement  could 
be  achieved  with  a  smaller  increase  in  cost  if  a  more  sophisticated  approach 
were  applied  by  relating  stockage  criteria  to  weapon  system  availability. 

e.  The  Repairables  Integrated  Model  for  Aviation  Material  (RIM-AIR), 
which  has  been  proposed  in  an  attempt  to  bring  AVCAL  policy  in  compliance  with 
DoD  policy  and  eliminate  the  inconsistency  of  current  policy,  is  a  step 
forward.  As  of  March  1983,  however,  none  of  the  RIM-AIR  additives  have 
actually  been  added  to  any  AVCAL,  and  the  model  is  subject  to  the  following 
shortcomings:137 

(1)  In  excluding  turnaround  times  for  beyond  capability  of 
maintenance  items,  it  underestimates  pipeline  spares  requirements. 

(2)  The  manner  in  which  the  Poisson  distribution  is  used  for 
computing  allowance  quantities  implicitly  assumes  an  infinite  repair  capacity, 
thereby  underestimating  pipeline  spares  requirements.  The  actual  repair 
process  is  nonhomogeneous . 

(3)  The  established  procedure  of  truncating  recorded  turnaround 
times  reduces  projected  turnaround  times  to  unrealistic  values,  again 
resulting  in  underestimating  pipeline  spares  requirements. 

(4)  As  a  depth  model,  it  must  be  supplemented  with  a  rule  for 
range;  optimization  must  be  used  to  determine  both  range  and  safety  level  for 
repairables  as  recommended  in  the  RIM-AIR  Study.133 


136CDR  James  L.  Bigby,  USN,  Shipboard  Parts  Allowance  Policy.  CNR  12 
(Alexandria,  Virginia:  Center  for  Naval  Analyses,  July  1981). 

137Mark  L.  Mitchell,  Retail  Level  Inventory  Model  for  Naval  Aviation 
Repairable  Items  (Monterey,  California:  Naval  Postgraduate  School,  March 
1983). 

138RIM-AIR  Study,  Report  155  (Mechanicsburg,  Pennsylvania:  Navy  Fleet 
Material  Support  Office,  June  1983). 
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(4)  The  policy  is  not  in  compliance  with  applicable  DoD  policy.132 

b.  Supply  support  management  is  inadequate  as  indicated  by  the 
following: 


(1)  With  a  gross  supply  effectiveness  goal  of  65  percent  (percent 
of  total  demands  satisfied  from  on-board  stock)  for  ship  material,  the  actual 
value  achieved  is  a  Fleet-wide  average  for  surface  ships  of  49  percent  in 
calendar  year  1979. 133 

(2)  With  an  (implicit)  allowance  effectiveness  goal  of  76  percent 
(percent  of  demands  that  are  for  items  included  in  the  allowance  list)  for 
ship  material,  the  actual  value  achieved  ranges  from  62  percent134  to  a 
three-year  average  of  40  percent.133 

(3)  In  spite  of  attempts  to  reduce  COSAL  costs,  only  eight  percent 
of  on-board  repair  parts  (Fleet-wide  average)  were  used  over  a  recent 
three-year  period,  with  the  remaining  92  percent  of  allowance  line  items 
unused  (see  Footnote  135). 

(4)  While  these  statistics  reveal  some  of  the  shortcomings,  more 
meaningful  measures,  such  as  the  number  or  percent  of  mission-degrading 
stock-outs,  are  not  monitored. 

c.  The  Fleet  Logistics  Support  Improvement  Program  (FLSIP)  model  used 
for  generating  COSALs  is  inadequate  for  the  following  reasons: 

(1)  It  is  designed  to  implement  stated  policy,  thereby  inhibiting 
relating  resources  to  readiness. 


132DoD  Directive  4140.44,  "Supply  Management  of  the  Intermediate  and 
Consumer  Levels  of  Inventory,"  February  1978. 

DoD  Instruction  4140.45,  "Standard  Stockage  Policy  for  Consumable 
Secondary  Items  at  the  Intermediate  and  Consumer  Levels  of  Inventory,"  April 
1978. 

DoD  Instruction  4140.46,  "Standard  Stockage  Policy  for  Repairable 
Secondary  Items  at  the  Intermediate  and  Consumer  Levels  of  Inventory,"  April 
1978. 

DoD  Instruction  4140.47,  "Secondary  Item  War  Reserve  Requirements 
Development,"  July  1979. 

133CDR  James  L.  Bigby,  USN,  Shipboard  Parts  Allowance  Policy.  CNR  12 
(Alexandria,  Virginia:  Center  for  Naval  Analyses,  July  1981). 

134Ibid. 


135Sarauel  D.  Judge  and  Palmer  Luetjen,  "Determination  of  Shipboard 
Repair  Parts  Level,"  Naval  Engineers  Journal  (April  1979),  pp.  37-43. 
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concluded  in  1981. 130  As  currently  proposed,  NTIPS  embodies  a  very 
ambit ious,  "cradle  to  grave,"  technical  information  management  system, 
encompassing  the  following  subsystems:  (1)  technical  information  requirements 
(for  job  and  training)  specifications,  (2)  information  generation  (computer- 
based  authoring)  using  LSA  data  and  equipment  source  data  as  direct  input,  (3) 
mastering/replication  (multimedia),  (4)  distribution,  a.id  (5)  delivery 
(multimedia).  All  five  subsystems  are  monitored  by  a  control  subsystem  which 
includes  configuration  management  and  deficiency  reports  processing  and 
disposition.  The  scope  of  NTIPS,  in  the  view  of  many  people,  is  too  grand  to 
have  a  chance  of  implementation.  7et,  it  appears  to  represent  the  right 
approach  to  solve  the  technical  information  problem.  A  great  deal  of  effort 
will  be  required  to  get  it  implemented.  Current  plans  call  for  implementation 
between  1986  and  1988,  with  full-scale  operation  in  1989.  A  limited  test  and 
evaluation  of  a  prototype  NTIPS  is  currently  in  progress. 

SUPPLY  SUPPORT 

Documentation  of  the  Navy's  supply  system,  stockage  criteria,  supply 
effectiveness  goals,  and  statistics  on  the  supply  achieved  in  the  Fleet  are 
beyond  the  scope  of  this  report.  The  following  comments  serve  to  summarize 
the  main  shortcomings  in  Navy  policy  (OPNAV  Instruction  4441. 12A,  "Supply 
Support  of  the  Operating  Forces,"  August  1973)  and  the  procedures  (models) 
used  in  constructing  the  Coordinated  Shipboard  Allowance  List  (COSAL)  and  the 
Aviation  Consolidated  Allowance  List  (AVCAL): 

a.  Current  stockage  policy  is  inadequate  for  four  reasons: 

(1)  The  stated  measures  of  effectiveness  governing  the  range  of 
items  stocked  (gross  and  net  supply  effectiveness)  do  not  relate  to  weapon 
system  readiness/sustainability,  the  primary  rationale  for  carrying  shipboard 
stocks. 


(2)  The  stated  measure  of  effectiveness  governing  the  depth  of 
items  stocked  also  does  not  distinguish  between  primary  mission-essential  and 
secondary  mission-support  items;  i.e.,  the  policy  treats  combat  systems  the 
same  as  laundry  machines. 

(3)  The  stated  aeronautical  material  availability  goals  cannot  be 
achieved  with  the  specified  stockage  criteria  stated  in  the  same  instruction 
as  demonstrated  by  previous  studies.131 


130NTIPS  Office,  Navy  Technical  Information  Presentation  System  (NTIPS) 
Final  Report  (Draft)  (Bethesda,  Maryland:  David  W.  Taylor  Naval  Ship  Research 
and  Development  Center,  January  1982). 

131RIM-AIR  Study,  Report  155  (Mechanicsburg,  Pennsylvania:  Navy  Fleet 
Material  Support  Office,  June  1983). 
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If  TM  management  is  improved  in  accordance  with  stated  policy,  problems 
experienced  in  the  past  can  be  avoided.  In  the  meantime,  inadequacies  of 
existing  THs  must  be  eliminated.  What  needs  to  be  done  is  clear. 

Requirements  for  verification  of  THs  should  be  satisfied,  not  waived.  Funding 
priorities  within  the  O&M's  Navy  budget  for  TH  management  should  be  increased 
to  eliminate  the  backlog  of  TM  revisions  and  updates.  The  following  policy 
issues  should  be  considered  to  improve  the  quality  of  THs: 

a.  Adoption  of  a  different  management  process,  using  maintenance 
channels  (similar  to  the  quarterly  force  revision  process  for  the  PNS)  vice 
technical  publications  channels. 

b.  Adoption  of  a  Navy  version  of  the  Air  Force  policy  giving  THs  the 
same  sanctions  as  direct  orders  (such  a  policy  obviously  would  require  high 
quality  THs) . 

Resident  training  courses  should  be  revised  to  alleviate  the  problems 
identified  by  the  Fleet  survey: 

a.  Devote  more  time  to  theory  of  operations.  In  current  courses,  time 
spent  per  TM  section  is  fairly  uniform.  Tet,  about  1  out  of  every  4 
technicians  report  difficulties  understanding  theory  of  operations,  while  only 
12  percent  have  difficulties  with  operating  procedures.12^ 

b.  Add  a  training  module  on  troubleshooting  formats,  especially  the 
FOMM  and  the  underlying  concept  of  "maintenance  dependencies." 

c.  Add  training  modules  for  all  common  test  equipment  that  technicians 
will  be  using  on  the  job.  Detailed  instructions  on  using  common  test 
equipment  are,  correctly,  not  included  in  prime  equipment  THs,  yet  80  to  90 
percent  of  school  graduates  lack  ability  in  test  equipment  use  and  expect  to 
find  procedural  instructions  in  THs. 

The  TH  Policy  Council  needs  to  promulgate  a  standard  specification  of  the 
minimum  prerequisites  a  TH  user  will  meet,  delineating  what  type  of 
information,  at  what  level  of  detail,  will  be  in  the  manual  and  what  skills, 
knowledge,  and  reading  comprehension  will  be  possessed  by  the  technician. 
Integration  of  training  and  technical  documentation  is  not  accorded  more  than 
lip  service  in  the  absence  of  such  a  specific  quantitative  standard. 

Long  term,  the  only  way  to  cope  with  the  explosive  growth  of  technical 
data  and  the  associated  quality  assurance  and  configuration  control  problems, 
is  complete  automation  of  the  generation,  distribution,  and  configuration 
control  of  technical  information.  That  was  the  original  thrust  of  the  Navy's 
R&D  effort  to  develop  NTIPS.  Over  the  years,  the  scope  of  NTIPS  has  grown  to 
include  the  interfaces  with  related  functional  areas.  The  design  phase  was 


129Ibid 
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b.  Develop  policy  and  responsibilities  for  the  institution  of  a 
Navy-wide  individual  proficiency  monitoring  system  for  enlisted  maintenance 
personnel.  The  lack  of  any  objective  measurement  of  individual  job 
proficiency  is  one  of  the  root-causes  of  the  Navy's  present  predicament.  It 
also  explains  the  disjoint  between  the  Fleet  and  the  training  establishment. 

c.  Expedite  R&D  in  critical  areas  that  are  not  receiving  adequate 
emphasis  in  current  R&D  programs,  with  the  following  areas  accorded  top 
priority: 


(1)  ISP.  Voids  in  task  analysis  and  training  know-how,  inadequate 
manuals,  and  lack  of  training  of  personnel  responsible  for  applying  ISD,  have 
resulted  in  serious  training-curriculum  deficiencies  and  a  poor  reputation  of 
ISD.  A  major  R&D  program  is  needed  to  correct  these  shortfalls.  Expanded 
task-analysis  procedures  must  be  developed  for  cognitive  tasks.  Procedural 
guidance  must  be  improved.  Computer-based  aids  to  ISD  must  be  developed.  A 
professional-development  program  for  ISD  practitioners  must  be  installed.  And 
the  whole  process  must  be  made  consistent  with  the  evolving  trend  to 
multilevel,  distributed  training. 

(2)  CAT.  The  CAT  R&D  program  is  designed  to  individualize  service 
applicant  testing  for  selection  and  classification  purposes  but  without 
changes  in  the  aptitude  test  battery  currently  used.  The  ASVAB,  however, 
measures  few  of  the  abilities  that  are  related  to  job-performance  potential. 
The  R&D  program  should  be  expanded  to  include  available  tests  of  cognitive 
abilities  predictive  of  job  performance. 

d.  Focus  management  attention,  force  decisions,  and  provide  oversight 
over  implementation  actions  regarding  the  following  issues: 

(1)  Occupational  Structure.  Because  the  recommendations  from  the 
NEOCS  study  were  never  acted  upon  (see  pages  71-72),  it  may  be  beneficial  to 
reopen  the  NEOCS  case,  give  it  a  full  hearing,  and  decide  on  the  best  course 
for  the  Navy.  The  current  occupational  structure  is  unmanageable, 
inefficient,  and  out  of  date.  These  shortcomings  also  detract  from  needed 
improvements  in  other  areas  (job  design,  training). 

(2)  Maintenance  Policy.  Current  policy  for  electronics  equipment 
installed  on  surface  combatants  is  ineffective.  Nodular  repair  capabilities 
must  be  authorized  aboard  ship.  This  capability  must  be  engineered  (test 
equipment,  technical  data,  authorized  parts,  skills),  not  left  out  as  an 
emergency  "band-aid"  as  in  the  current  2-K  program.  The  current  policy  will 
become  effective  only  when  testability  has  been  improved  and  shipboard 
stockage  policies  revised. 

(3)  Integrated  Diagnostics.  Management  action  is  required  to 
implement  the  integrated  diagnostics  concept.  Current  policy  (instructions) 
must  be  revised,  guides  must  be  promulgated,  Navy  input  to  new  and  revised 
military  standards  must  be  coordinated,  and  contract  item  specifications  must 
be  revised. 
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(4)  Maintenance  Data  Automation.  The  thrust  of  the  SNAP  II  program 
(relieving  shipboard  administrative  burdens  serving  the  interests  of  the  shore 
establishment)  and  the  associated  physical  equipment  (stand-alone  computers) 
must  be  reconsidered.  Shipboard  requirements  would  be  better  served  by  a  dis¬ 
tributed  data  processing  capability  with  application  programs  designed  to  serve 
the  needs  of  the  ship,  including  maintenance  information  for  the  technicians. 

(5)  Support  of  Electronic  Aids.  Shipboard  maintenance  will,  in  the 
future,  increasingly  rely  on  electronic  aids.  Current  plans  are  that  the  same 
technicians,  who  will  use  these  aids  to  perform  maintenance  on  prime 
equipment,  will  also  maintain  these  aids.  The  soundness  of  this  support 
concept  is  in  doubt.  A  special  rating  (such  as  the  TD  rating  which  was 
recently  disestablished  by  lack,  of  sea  billets)  to  support  all  shipboard 
electronic  aids  may  be  a  better  alternative.  A  policy  for  the  support  of 
electronic  aids  must  be  established  now,  while  it  is  still  early  enough  to 
influence  the  design  characteristics  of  these  aids. 

e.  Provide  sponsorship  and  resources  for  the  specific  R&D 
recommendations  in  maintenance  training  and  aiding  technology  specified  in  the 
remainder  of  this  section: 

(1)  Embedded  maintenance  training. 

(2)  Embedded  maintenance  aids. 

(3)  Job  performance  and  training  aids. 

(4)  Maintenance  training  equipment. 

A  formal  cost-benefit  analysis  has  not  been  conducted  in  justification  of 
these  recommendations.  It  is  believed  that  the  Maintenance  Improvement  Plan 
could  be  executed  at  a  small  fraction  of  the  true  cost  associated  with  poor 
maintenance  —  a  cost  which  is  estimated  to  run  into  the  billions  if  equipment 
redundancies,  necessitated  by  poor  maintenance,  are  included. 

EMBEDDED  MAINTENANCE  TRAINING 

Both  the  Defense  Science  Board's  1982  Summer  Study  and  the  Navy  Training 
Improvement  Program  address  the  need  to  develop  "embedded"  or  "strap-on" 
training  capabilities  for  new  weapon  systems  with  embedded  computer 
resources.  The  focus  of  past  and  current  research  efforts  has  been  directed 
primarily  at  operator  training  capabilities one  example  of  equipment 


1a9No  information  about  current  research  projects  on  embedded  training 
which  may  have  been  sponsored  recently  by  the  Navy  or  the  Air  Force  was 
available.  DARPA  has  no  research  program  in  embedded  training,  in  spite  of 
announcements  to  the  opposite  effect.  The  Army  Research  Institute  (ARI)  is 
sponsoring  a  research  study  on:  "System  Design  Concepts  to  Support  Embedded 
Training"  (Request  for  Proposals,  Commerce  Business  Daily,  11  August  1983). 
ARI  could  not  provide  further  details  on  its  research  plans  because  the 
information  is  "procurement  sensitive." 
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stimulation  for  training  (not  embedded  training)  is  the  FFG-7  Class  Pierside 
Combat  System  Trainer.  It  is  scheduled  for  delivery  in  early  1984  and  will  be 
used  for  shipboard  combat  system  operator  and  team  training.  Evaluation  of 
this  trainer  is  intended,  inter  alia,  to  provide  information  on  the  most 
economical  way  for  integrating  this  training  capability  within  future  combat 
systems.  (In  the  case  of  the  FFG-7  Class,  embedding  this  capability  within 
the  existing  combat  system  will  not  be  economical.) 

It  does  not  appear  that  an  embedded  maintenance  (as  opposed  to  operator) 
training  concept  would  be  practical  since  embedded  computer  resources  are 
sized  for  prime  mission  requirements;  additional  requirements  for  maintenance 
training  (software  for  simulation  and  stimulation  and  courseware)  would  affect 
the  reliability  and  maintainability  characteristics  of  the  prime  equipment. 
Instead,  a  "strap-on"  concept  (hooking  up  a  general  purpose  mini-  or 
microcomputer  to  the  prime  equipment)  would  have  significant  potential.  Prime 
contractors  may  already  be  developing  much  of  the  required  software  for  the 
purpose  of  BIT  design  evaluation,  though  the  software  is  not  currently  a 
contract  deliverable  item.  Some  past  contracts  (e.g.,  the  F-15  in  the  Air 
Force)  included  a  requirement  for  simulating  system  failures  for  assessing  BIT 
performance  as  part  of  the  formal  maintenance  demonstration  and  development 
test  requirements.  Vith  the  increasing  emphasis  on  testability  as  a  key 
maintainability  characteristic,  such  simulation  requirements  can  be  expected 
to  become  a  standard  feature  in  future  weapon  system  acquisition  contracts. 

The  reason  is  that  this  is  the  only  way  that  BIT  performance  can  be  assessed 
adequately.  The  important  point  is  that  the  same  software  (failure-simulation 
model)  can  be  applied  in  a  training  mode  to  teach  the  technician  the  BIT 
indications  on  the  prime  equipment,  the  extent  of  false  alarms  and 
ambiguities,  and  the  fault-isolation  procedures  required  to  find  the  simulated 
failure. 

Needless  to  say,  this  would  be  the  most  cost-effective  approach  to 
troubleshooting  training  with  a  maximum  transfer-of-training,  and  offering  the 
potential  of  eliminating  most  of  the  actual  equipment  trainers  from  "C" 
Schools.  (There  would  still  be  a  need  for  part-task  trainers  at  those  school 
to  train  remove/replace  actions,  system  knowledge,  nomenclature,  etc.  Also, 
this  shipboard  troubleshooting  training  would,  of  course,  be  limited  to  those 
failures  covered  in  BIT  design;  additional  training  may  be  required  later  to 
learn  to  cope  with  other  failure  modes,  external  influences  such  as  damage, 
etc.)  Uhat  needs  to  be  done  to  bring  this  concept  to  fruition  appears  to  be 
very  limited: 

a.  The  prime-equipment  contract  must  include  a  requirement  for  an 
interface  adapter,  permitting  external  computer-controlled  stimulation  of  the 
prime  equipment.  A  standard  specification  should  be  developed  for  such  an 
adapter. 


b.  The  prime-equipment  contract  must  include,  as  a  contract 
deliverable,  the  failure-simulation  model  developed  by  the  contractor  in 
designing  the  BIT  and  measuring  BIT  performance. 
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c.  Conventional  computer-based  courseware  must  be  developed  around  the 
failure-simulation  model  to  interface  with  the  trainee-technician  via  a 
separate  display. 

It  is  recommended  that  this  approach  be  examined  by  the  Navy's 
engineering  laboratories,  not  the  NAVTRAEQUIPCEN  or  the  NAVPERSRANDCEN. 
Consequently,  this  project,  with  the  most  immediate  and  highest  payoff  of  the 
R&D  recommendations,  is  not  included  in  Appendix  A. 

EMBEDDED  MAINTENANCE  AIDS 

There  is  a  need  for  more  attention  to  the  technician's  information 
requirements  in  designing  built-in  maintenance  diagnostics.  While  much  has 
been  written  on  BIT  design,  testability  requirements  and  how  to  measure  it, 
and  troubleshooting  procedures,  the  way  in  which  BIT  information  is  to  be 
displayed  has  been  accorded  little  attention.  There  are  no  specifications  or 
design  guides,  to  our  knowledge,  that  address  this  issue  in  a  meaningful  way. 
Thus,  prime  contractors  are  given  much  leeway  in  this  respect,  with  the  result 
that  BIT  indications  sometimes  leave  a  lot  to  be  desired,  apart  from  the  issue 
of  BIT  performance  itself.  For  example,  one  might  conceive  of  a  way  to  reduce 
the  impact  of  misleading  BIT  indications  (false  alarms,  erroneous  fault 
isolations,  or  excessive  ambiguity  groups)  by  requiring  the  BIT  also  to 
display  on  request,  those  areas  of  the  system  it  has  tested  to  be  good,  and 
those  areas  that  are  marginal.  Or.  for  airborne  equipment,  one  might  insist 
on  a  capability  to  eliminate  BIT  warnings  resulting  from  intermittent 
failures.  This  study  did  not  allow  a  detailed  analysis  of  this  issue,  but  it 
seems  that  this  area  needs  more  research. 

It  is  recommended  that  the  Navy's  design  for  testability  R&D  program  be 
expanded  to  address  the  subject  of  how  BIT  indications  should  be  displayed  or 
processed  for  maximum  maintenance  effectiveness.  Again,  this  project  concerns 
the  prime  equipment,  not  maintenance  training  equipment  or  stand-alone  aids. 

It  is,  therefore,  not  included  in  Appendix  A. 

Another  type  of  built-in  aid  is  what  is  commonly  referred  to  as 
display-aided  maintenance;  i.e.,  the  job  aid  (telling  the  technician  what  to 
do  or  where  to  look  in  his  manual)  is  built  into  the  prime  equipment,  so  to 
speak.  (Physically,  display-aided  maintenance  is  invariably  implemented  on  a 
separate  computer,  not  part  of  the  embedded  computer  resources  supporting  the 
tactical  software  for  the  very  reason  mentioned  in  the  previous  section. 
Because  this  separate  computer  is  part  of  the  weapon  system,  it  is  counted 
among  the  embedded  computer  resources,  but  there  is  a  difference.)  This 
approach  will  become  more  common  in  the  future  for  large,  integrated  systems. 
Much  of  the  job-aiding  technology  being  developed  (or,  to  be  developed)  with  a 
view  to  stand-alone  aids  is  directly  applicable  to  such  built-in  aids;  e.g., 
what  information  to  display  and  how  to  display  it.  There  is  currently  a  lack 
of  communication  between  the  prime-equipment  developers  and  the  job-aiding 
technology  developer.  This  limits  reaping  the  full  benefits  from  R&D 
investments  in  job-aiding  technology.  The  approach  for  correcting  this 
problem  lies  with  the  development  of  specifications  and  guides  for  job-aiding 
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software  addressed  in  the  next  section  and  summarized  as  B&D  Project  3 
(Appendix  A). 

JOB  PERFORMANCE  AND  TRAINING  AIDS 

In  the  area  of  job  performance  aids,  additional  research  should  be 
focused  on  answering  the  following  two  questions: 

a.  How  do  technicians  perform  maintenance  and  what  information  (in  what 
form,  under  which  conditions,  in  what  sequence)  do  they  need  to  minimize 
maintenance  errors  and  maximize  maintenance  effectiveness  and  efficiency? 

b.  What  distinguishes  exactly  an  expert  troubleshooter  from  an  average 
technician? 

A  detailed  investigation  of  the  first  question  is  necessary  for  the 
design  of  more  effective  aiding  software.  The  second  question  needs  to  be 
examined  more  thoroughly  than  has  been  done  in  the  past  in  order  to  acquire 
the  knowledge  necessary  to  design:  (1)  more  effective  training  approaches  for 
the  development  of  troubleshooting  skills,  and  (2)  aiding  software 
enhancements  to  maximize  learning  from  using  the  aid.  The  experiments 
necessary  to  pursue  these  two  questions  are  listed  in  Appendix  A.  Some  care 
must  be  applied  in  selecting  the  systems  used  in  these  experiments.  In 
practice  this  may  not  be  a  matter  of  choice,  but  depends  on  cooperation  from 
the  Fleet  or  Systems  Command  program  (project)  managers. 

Electronic  maintenance  aids  enhanced  with  tutorial  software  cannot  cover 
all  the  training  requirements.  As  this  study  indicates,  there  is  a  serious 
need  for  additional  training  in  the  Fleet.  This  training  requirement, 
currently  being  met,  in  part,  through  local  initiatives  using  workaround 
procedures,  is  bound  to  grow  in  the  future  as  a  result  of  increasing  pressures 
to  reduce  front-end  resident  school  training.  Shipboard  technical  skill 
training,  apart  from  OJT  by  supervisors,  entirely  depends  on  the  printed 
medium  and  individual  motivation.  The  quality  of  this  type  of  nonresident 
school  training  would  benefit  enormously  from  a  more  disciplined  approach, 
including  a  well-designed  training  delivery  system  for  shipboard  and 
"waterfront"  use.  It  is  recommended  that  the  Navy  initiate  a  major  research 
program  focused  on  meeting  this  need: 

a.  Identify  the  technical  training  requirements  in  the  Fleet. 

b.  Resurrect  the  aborted  PPAS  program  as  a  viable  approach  to 
individual  skill-proficiency  assessment,  and  continue  its  development  and 
testing  (it  appears  that  the  Fleet's  attitude  has  changed  in  that  the  need  for 
a  more  disciplined  approach  to  proficiency  testing  and  certification  is 
increasingly  being  recognized). 

c.  Design  the  most  cost-effective  training  delivery  system  to  meet 
Fleet  training  requirements,  including  administration  of  job  proficiency  tests 
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and  conduct  of  remedial  training  at  TYCOM  training  activities  as  well  as 
shipboard  skill-enhancement  training. 

d.  Develop  training  modules  based  on  "expert  system"  principles,  using 
the  information  gained  from  the  experiments  recommended  earlier. 

e.  Test  and  evaluate  the  prototype  training  modules  (for  selected 
skills,  specifically  troubleshooting,  for  selected  systems)  installed  on 
prototypes  of  the  selected  training  delivery  system. 

The  R&D  in  job  performance  and  training  aids  should  be  accorded  highest 
priority  in  view  of  anticipated  payoffs  in  improved  maintenance  performance. 

In  recognition  of  the  urgency  of  meeting  Fleet  training  requirements,  the  Navy 
should  set  a  formal  target  date  in  the  late  1980s  for  a  fully- implemented, 
operational  training  delivery  system  encompassing  the  task/skill/knowledge 
data  base,  proficiency  tests,  remedial  courseware,  interactive  terminals  for 
individualized  learning  (computer-aided  instruction  (CAI)),  skill  enhancement 
courseware,  and  training-enhanced  electronic  maintenance  aids.  The  target 
date  is  important  because  by  the  late  1980s  this  type  of  system  will  not  be  a 
luxury,  but  a  necessity  as  a  result  of  the  known  personnel  trends. 

MAINTENANCE  TRAINING  EQUIPMENT 

There  are  serious  voids  of  knowledge  in  virtually  all  areas  related  to 
the  acquisition  and  use  of  training  equipment  which  is  both  training-effective 
(high  transfer-of-training)  and  cost-effective.  The  need  for  more  research  in 
these  areas  was  identified  over  ten  years  ago  and  much  research  has  been  done 
in  the  meantime;  it  is  clear  that  the  issues  involved  are  complex.  To 
maximize  payoffs  from  available  research  budgets,  it  is  recommended  that  the 
Navy  pursue  an  interservice  research  program  approach,  coordinating  its  own 
efforts  by  the  NAVTRAEQUIPCEN  and  the  NAVPERSRANDCEN  with  those  of  the  other 
Services,  the  Air  Force  Human  Resources  Laboratory  (AFHRL)  and  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI),  which  focuses 
on  the  following: 

a.  Development  of  a  new  job-task  analysis  procedure  for  the  exclusive 
purpose  of  identifying  functional  requirements  for  training  devices. 

b.  Investigation  of  the  relationships  between  fidelity  (physical, 
functional,  and  psychological),  training  objectives,  and  transfer-of-training 
in  the  context  of  maintenance  training  equipment. 

c.  Development  of  a  modular  concept  for  training  simulators, 
standardizing  the  various  modules  that  constitute  a  trainer. 

d.  Development  of  a  Training  Equipment  Acquisition  Manual  as  a  decision 
aid  to  assist  in  establishing  training-device  requirements  for  new  weapon 
system  acquisitions: 

(1)  Validation  of  the  need  for  new  trainers. 


116 


NAVTBAEQUIPCEN  MDA903-81-C-0166-1 


(2)  Identification  of  the  mix  of  trainers  required  (based  on 
knowledge  gained  from  (a)  and  (b)  above). 

(3)  Determination  of  the  functional  characteristics  required  of 
each  new  trainer  or  software/hardware  changes  in  existing  trainers. 

(4)  Specification  of  standard  procedures  for  acquiring  the  needed 
trainers  (including  solicitation,  contract  award,  evaluation,  and  production). 

e.  Documentation  of  lessons  learned  from  existing  training  devices  and 
institution  of  a  standard  operating  procedure  for  follow  up  of  each  new 
device.  This  documentation,  in  combination  with  a  continuously  updated 
register  of  the  inventory  of  all  maintenance  training  equipment  (including 
their  key  characteristics),  should  be  promulgated  as  an  annex  to  the 
Acquisition  Manual. 

f .  Development  of.  a  series  of  guides  to  help  Fleet  Project  Team 
personnel  do  a  better  job: 

(1)  Job-Task  Analysis  Guide  (based  on  “a."  above). 

(2)  School  Introduction  Guide. 

(3)  Training  Equipment  Evaluation  Guide. 

g.  Development,  installation,  and  test  of  troubleshooting  courseware  in 
BEE  School  using  video/computer-graphics,  using  the  standard  "Souse  package" 
(context-free  and  context-specific),  supplemented  with  a  third  stage  depicting 
general  functional  relationships  of  "typical"  electronic  systems.  (Ensure 
that  the  software  is  designed  to  catch  ineffective/inappropriate  behaviors 
before  they  practiced  and  reinforced). 

h.  Examination  of  the  wide  variety  of  technology  trainers  currently 
used  in  BEE  and  "A"  Schools  in  order  to  standardize  on  a  small  group  for 
Navy-wide  (not  school-by-school)  application,  including:  basic  circuit 
trainers,  special  circuit  trainers,  servo-mechanism  trainers,  and  digital 
logic  trainers. 

i.  Development  of  design  specifications  for  generalized  training 
devices  for  use  in  "A"  Schools,  followed  by  acquisition  and  testing  of 
prototypes.  Host  of  the  current  equipment  is  either  obsolete  or  ineffective. 
This  effort  should  include  one  of  each  of  the  following: 

(1)  Generalized  Transceiver  Trainer. 

(2)  Generalized  Badar  Maintenance  Trainer. 

(3)  Generalized  Computer/Peripherals  Maintenance  Trainer. 


117 


NAVTRAEQUIPCEN  MDA903-81-C-0166-1 


(4)  Generalized  Fire  Control  Maintenance  Trainer  (i.e.,  restore 
funding  of  device  11D20). 

(5)  Generalized  IFF  Maintenance  Trainer. 

(6)  Generalized  Communications  Maintenance  Trainer. 

(7)  Generalized  Sonar  Maintenance  Trainer  (i.e.,  restore  funding  of 
device  14E21) . 

j.  Development  and  installation  of  self-paced  course  modules  in  BEE 
School  to  teach  setup  and  use  of  general-purpose  test  equipment  (using  CAI, 
not  CMI) . 

The  corresponding  projects  are  listed  in  Appendix  A.  Neither  the  levels 
of  research  effort  required  for  these  projects,  nor  their  relative  importance 
or  payoff  are  estimated.  All  would  likely  contribute  to  improved  maintenance 
training.  The  other  Services  have  ongoing  research  projects  in  some  of  these 
areas.  Coordination  among  the  Services  clearly  is  required  to  avoid 
duplication  of  work. 

MONITORING  TECHNOLOGY  IMPACTS 

Finally,  it  is  important  to  highlight  one  more  issue:  the  need  for  some 
organization  in  the  Navy  to  be  responsible  for  closing  the  gap  between  what  is 
taught  in  Navy  "A"  Schools  (training  content)  and  the  maintenance  skills 
required  in  the  Fleet  as  a  result  of  the  technology  employed.  The  two 
existing  feedback  mechanisms  which  are  intended  to  close  the  gap  (see  page  84) 
are  in  place,  however,  the  success  has  been  somewhat  limited.  Consequently, 
that  gap  is  still  very  large.  For  example,  the  BEE  course  content  is, 
essentially,  a  programmed-instruction  adaptation  of  a  1960s-era  course  for 
electronics  hobbyists;  apart  from  some  basic  electronics  fundamentals,  much  of 
the  course  has  little  relationship  to  today's  first-term  technician  skill 
requirements;  and,  as  of  1983,  no  digital  fundamentals  are  taught.  An  "A" 
School  example  is  the  revision  of  "A"  School  courses  feeding  "C"  Schools  for 
th  Navy  Tactical  Data  System  (NTDS) .  This  revision  was  initiated  in  1983, 
though  NTDS  was  first  installed  nearly  20  years  ago.  While  it  is  true  that, 
at  any  point  in  time,  the  technology  employed  in  the  Fleet  may  cover  a  span  of 
some  20  years,  the  current  gap  between  training  course  content  and  technology 
is  more  than  is  necessary  or  desirable. 

Changes  in  combat  system  technology  are  rapid  and  the  rate  of  change  can 
be  expected  to  accelerate  in  the  future.  The  assessment  of  future  technology 
impacts  on  maintenance  skill  and  training  requirements  was  very  limited  and 
considered  only  some  very  general  trends.  Nevertheless,  the  impact  will  be 
significant.  Rather  than  allowing  training  to  lag  technology  by  some  20 
years,  the  Navy  should  strive  to  stay  abreast.  It  is  recommended,  therefore, 
that  an  office  be  created  in  Headquarters,  Naval  Material  Command,  responsible 
for  monitoring  technology  trends,  assessing  the  potential  impacts  on 
personnel,  training,  and  skill  requirements,  and  disseminating  this 


118 


NAVTRAEQUIPCEN  HDA903-81-C-0166-1 


information  among  those  Navy  decision  makers  responsible  for  these  functional 
areas.  The  office,  to  be  effective,  should  be  represented  es  a  permanent 
member  on  NAVNAT's  Acquisition  Review  Group  and  Logistics  Review  Group. 
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SECTION  VI 
POSTSCRIPT 


The  focus  of  this  study  and  the  resulting  R&D  recommendations  have 
addressed  what  needs  to  be  done  to  improve  the  acceptance,  utilization,  and 
effectiveness  of  maintenance  job  aids  and  maintenance  training  equipment. 
Recommendations  in  other  areas  are  limited  to  those  which  interact  directly 
with  this  main  theme.  It  is  acknowledged  that  we  have  only  scratched  the 
surface  in  many  related  areas,  and  that  much  more  R&D  will  be  necessary  to 
support  a  fully-effective  Maintenance  Improvement  Plan. 
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ACTS 

AD 

ADP 

AFHRL 

AFQT 

AFTA 

AI 

AIMD 

AMTESS 

Ao 

AQ 

AR 

ARI 

AS 

ASVAB 

AT 

ATE 

AVCAL 

AX 


LIST  OF  ABBREVIATIONS,  ACRONYMS ,  AND  SYMBOLS 


Adaptive  Computer  Training  System 

destroyer  tenders 

automated  data  processing 

Air  Force  Human  Resources  Laboratory 

Armed  Forces  Qualification  Test 

Avionics  Fault  Tree  Analyzer 

artificial  intelligence 

Aircraft  Intermediate  Maintenance  Department 

Army  Maintenance  Training  and  Evaluation  Simulation  System 

operational  availability 

Aviation  Fire  Control  Technician 

repair  ships 

Army  Research  Institute 

submarine  tenders 

Armed  Services  Vocational  Aptitude  Battery 
Aviation  Electronics  Technician 
automated  test  equipment 
Aviation  Consolidated  Allowance  List 
Aviation  Antisubmarine  Warfare  Technician 


Basic  Electricity  and  Electronics 

built-in  test 

built-in  test  equipment 


CAD 

CAI 

CAPT 

CASREP 

CAT 

CDR 

CINCLANTFLT 

CLASP 

CMI 

CNET 

CNO 

CNR 

COMNA VS  URFLANT 

COMNAVSURFPAC 

COSAL 

CPU 

CSIP 

CSMP 

CTM 

CV 

CVN 


computer-aided  design 
computer-aided  instruction 
Captain 

Casualty  Report 

computerized  adaptive  testing 

Commander 

Commander  in  Chief,  U.S.  Atlantic  Fleet 

Classification  and  Assignment  with  PRIDE 

computer-managed  instruction 

Chief  of  Naval  Education  and  Training 

Chief  of  Naval  Operations 

Chief  of  Naval  Research 

Commander,  Naval  Surface  Force,  U.S.  Atlantic  Fleet 

Commander,  Naval  Surface  Force,  U.S.  Pacific  Fleet 

Coordinated  Shipboard  Allowance  List 

central  processing  unit 

Combat  Systems  Improvement  Program 

Current  Ship  Maintenance  Project 

Cryptologic  Technician,  Maintenance 

aircraft  carrier 

aircraft  carrier  (nuclear) 


121 


NAVTRAEQUIPCEN  MDA903-81-C-0166-1 


"D" 

DARPA 

DoD 

DS 

DSARC 

DSB 

EDVR 

EEMT 

EPICS 

ET 

EU 

FCS 

FIND 

FLSIP 

FMC 

FOMM 

FPT 

FSD 

FT 

FTB 

FTG 

FTM 

FY 

GAO 

GFCS 

GI 

GM 

GMM 

GUTS 

HF 

HFE 

HSDG 

HUD 

Hz 


IBM 

IC 

IDA 

IFF 

ILS 

ISD 


-  depot 

-  Defense  Advanced  Research  Projects  Agency 

-  Department  of  Defense 

-  Data  Systems  Technician 

-  Defense  Systems  Acquisition  Review  Council 

-  Defense  Science  Board 

-  Enlisted  Distribution  Verification  Report 

-  Electronic  Equipment  Maintenance  Trainer 

-  Enlisted  Personnel  Individualized  Career  System 

-  Electronics  Technician 

-  Electronics  Warfare  Technician 

-  Fire  Control  System 

-  Fault  Isolation  through  Nodal  Dependencies 

-  Fleet  Logistics  Support  Improvement  Program 

-  fully  mission  capable 

-  Functionally  Oriented  Maintenance  Manual 

-  Fleet  Project  Team 

-  full-scale  development 

-  Fire  Control  Technician 

-  Ballistic  Missile  Fire  Control  Technician 

-  Fire  Control  Technician  (Gun  Fire  Control) 

-  Fire  Control  Technician  (Missile  Fire  Control) 

-  fiscal  year 

-  General  Accounting  Office 

-  Gun  Fire  Control  System 

-  soldier 

-  Gunners 1  Mate 

-  Gunner's  Mate  (Missiles) 

-  Generalized  Maintenance  Training  Simulator 

-  high  frequency 

-  human  factors  engineering 

-  High  School  Diploma  Graduates 

-  head-up  display 

-  hertz 

-  intermediate 

-  International  Business  Machines  Corporation 

-  integrated  circuit 

-  Institute  for  Defense  Analyses 

-  identification  friend  or  foe 

-  integrated  logistics  support 
instructional  systems  development 


JCS 

JLC 

JPA 


Joint  Chiefs  of  Staff 
Joint  Logistics  Commanders 
Job  Performance  Aid 
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LCC 

LCDR 

LHA 

LMI 

LOGMOD 

LOGPAC 

LRU 

LSA 

LSSD 

LT 

H 

HADT 

HC 

HDS 

MDT 

MIL-HDBK 

HIL-STD 

MIP 

Hk 

HLDT 

HOTU 

HRC 

H.S. 

HT 

HTBH 

HTTR 

N/A 

NADC 

NAMTRADET 

NASA 

NATO 

NATSF 

NAVAIR 

NAVEDTRA 

NAVELEX 

NAVHAT 

NAVPERS 

NAVPERSRANDCEN  - 

NAVSEA 

NAVTIP 

NAVTRAEQUIPCEN  - 

NEC 

NEOCS 

NEPDIS 

NFR 

NIMS 

NMC 

NHCM 


Amphibious  Command  Ships 
Lieutenant  Commander 
Amphibious  Assault  Ships 
Logistics  Management  Institute 
logic  model 

Logistics  Command,  Pacific 

lowest  replaceable  unit  (also  line  replaceable  unit) 
logistics  support  analysis 
Level  Sensitive  Scan  Design 
Lieutenant 

million 

mean  administrative  delay  time 

Mental  Category  or  mission  capable 

Maintenance  Data  System 

mean  down  time 

military  handbook 

military  standard 

Maintenance  Index  Page 

Hark 

mean  logistics  delay  time 

Mobile  Technical  Unit 

Maintenance  Requirements  Card 

Master  of  Science 

Missile  Technician 

mean  time  between  maintenance 

mean-time-to-repair 

not  applicable 

Naval  Air  Development  Center 

Naval  Air  Maintenance  Training  Detachment 

National  Aeronautics  and  Space  Administration 

North  Atlantic  Treaty  Organization 

Naval  Air  Technical  Services  Facility 

Naval  Air  Systems  Command 

Naval  Education  and  Training 

Naval  Electronic  Systems  Command 

Naval  Material  Command 

Naval  Personnel 

Navy  Personnel  Research  and  Development  Center 

Naval  Sea  Systems  Command 

Navy  Training  Improvement  Program 

Naval  Training  Equipment  Center 

Naval  Enlisted  Classification 

Navy  enlisted  occupational  classification  system 

Naval  Enlisted  Professional  Development  Information  System 

no-fault  removal 

NATSF  Information  Management  System 

not  mission  capable 

not  mission  capable  maintenance 


123 


NAVTRAEQUIPCEN  MDA903-81-C-0166-1 


NMCS 

NMP 

NRAC 

NSIA 

NTDS 

NTIPP 

NTIPS 

"0" 

OJT 

O&H 

ONR 

OPNAV 

ORTS 

OSD 

PAR 

PEAK 

PMC 

PM/FL 

PMS 

PO 

PPAS 

PQS 

PRIDE 

R&D 

RADN 

R&M 

RFT 

RIM-AIR 

RM 

SAT 

SATCOM 

SECDEF 

SERT 

SIMA 

SIMSPO 

SNAP 

SRA 

SRF 

ST 

STAMP 

STEEP 

STG 

STS 

SUBACS 


-  not  mission  capable  supply 

-  Navy  Hanning  Plan 

-  Naval  Research  Advisory  Committee 

-  National  Security  Industrial  Association 

-  Naval  Tactical  Data  System 

-  Navy  Technical  Information  Presentation  Program 

-  Navy  Technical  Information  Presentation  System 

-  organizational 

-  on-the-job  training 

-  operating  and  maintenance 

-  Office  of  Naval  Research 

-  Office  of  the  Chief  of  Naval  Operations 

-  Operational  Readiness  Test  System 

-  Office  of  the  Secretary  of  Defense 

-  Personnel  Advancement  Requirement 

-  Personal  Electronic  Aid  for  Maintenance 

-  partially  mission  capable 

-  performance  monitoring/fault  localization 

-  planned  maintenance  system 

-  Petty  Officer 

-  personnel  proficiency  assessment  system 

-  Personnel  Qualification  Standards 

-  personalized  recruiting  for  immediate  and  delayed  enlistment 

-  research  and  development 

-  Rear  Admiral 

-  Reliability  and  Maintainability 

-  ready-for-training 

-  Repairables  Integrated  Model  for  Aviation  Material 

-  Radioman 

-  Scholastic  Aptitude  Test 

-  satellite  communications 

-  Secretary  of  Defense 

-  Ship  Electronics  Readiness  Team 

-  shore  intermediate  maintenance  activity 

-  Simulator  Systems  Program  Office 

-  Shipboard  Non-tactical  ADP  Program 
shop  replaceable  assembly 

-  Ship  Repair  Facility 

-  Sonar  Technician 

-  System  Testability  and  Maintainability  Program 

-  Support  and  Test  Equipment  Engineering  Program 

-  Sonar  Technician  (Surface) 

-  Sonar  Technician  (Submarine) 

-  Submarine  Advanced  Combat  System 
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PROJECT  11:  DEVELOPMENT  OF  SPECIFICATIONS  FOR  MAINTENANCE  TRAINING  EQUIPMENT 
(NON-SYSTEM  SPECIFIC) 

BACKGROUND.  Once  the  Navy  has  decided  upon  a  training  strategy  and  the 
implications  for  training  simulators  have  been  considered  (Project  10), 
specifications  must  be  developed  for  that  equipment.  Normal  task,  analysis 
does  not  suffice  for  this  purpose,  even  after  revised  procedures  have  been 
developed  for  training  device  specification  (Project  7),  because  the  training 
equipment  is  not  tied  to  specific  end-item  tasks.  Non-system-peculiar  devices 
include  technology  trainers  (simulating  one  or  more  sections  of  electronic 
equipment  to  illustrate  operational  or  technological  principles);  generalized 
trainers  (simulating  the  characteristics  of  a  class  or  family  of  equipments  to 
teach  theory  of  operations,  functional  relationships,  testing,  measurements, 
and  troubleshooting);  and  general-purpose  trainers  (which,  through 
system-peculiar  software  and  audiovisuals,  can  be  made  to  simulate  any 
specific  system).  This  project  addresses  requirements  for  the  first  two 
categories . 

TASK  DESCRIPTION. 

a.  Review  the  requirements  for  technology  trainers  at  the  various 
schools.  Define  the  commonalities  among  these  requirements.  Hatch  the 
characteristics  of  current  devices  to  these  requirements.  Identify  what  is 
obsolete  and  what  is  duplicative.  Standardize  on  the  smallest  group  of 
technology  trainers  possible  for  Navy-wide  application: 

(1)  Basic  circuit  trainers. 

(2)  Special  circuit  trainers. 

(3)  Servo-mechanism  trainers. 

(4)  Digital  logic  trainers. 

b.  Institutionalize  a  periodic  review  process  of  technology  trainer 
requirements . 

c.  Develop  design  specifications  for  generalized  trainers: 

(1)  Transceiver  trainer. 

(2)  Radar  maintenance  trainer. 

(3)  Computer/peripherals  maintenance  trainer. 

(4)  Search  radar/fire  control  maintenance  trainer. 

(5)  IFF  maintenance  trainer. 
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PROJECT  10:  DEVELOPMENT  OF  A  STANDARDIZED  APPROACH  TO  TROUBLESHOOTING  TRAINING 

BACKGROUND.  Past  research  suggests  that  there  is  substantial  benefit  in  first 
training  generic  troubleshooting  skills  and  then  teaching  troubleshooting, 
using  2-D  simulators,  on  classes  of  equipment  or  specific  equipment  to  address 
equipment-peculiar  features.  Versatility  has  been  shown  to  be  the  most 
important  aspect  of  troubleshooting  skills  in  view  of  the  continuous 
engineering  changes  or  improvements  of  military  equipment.  This  is  why 
teaching  troubleshooting  skills  on  classes  of  equipments  should  receive  more 
emphasis  in  lieu  of  troubleshooting  specific  equipments.  To  guide  the 
development  of  the  appropriate  training  equipment,  an  overall  Navy-wide 
approach  must  be  developed.  Apparently,  this  has  not  been  done.  While  we 
have  definite  ideas  on  this  subject  (see  Project  5),  this  is  based  more  on 
intuition  than  quantitative  evaluation  of  alternatives. 

TASK  DESCRIPTION 

a.  Determine  alternative  options  for  developing  troubleshooting  skill 
proficiency  of  combat  system  technicians. 

b.  Evaluate  the  pros  and  cons  of  the  alternatives,  including 
acquisition  and  support  cost  estimates. 

c.  Decide  what  alternative  is  best  on  the  basis  of  training¬ 
effectiveness  and  cost-effectiveness. 

NOTES . 

a.  Training  strategy  is  closely  interlinked  with  this  issue  and  that 
addressed  in  Project  9.  The  current  training  strategy  would  favor  technology 
trainers  in  BEE  School,  generalized  trainers  in  "A"  Schools,  and  general- 
purpose  trainers  in  "C"  Schools.  The  implications  of  EPICS  are  that  the  first 
training  phases  are  more  equipment-oriented,  so  that  the  need  for  general- 
purpose  trainers  (with  subsystem/system-peculiar  software)  is  deferred  to 
subsequent  training  phases,  and  generalized  trainers  may  be  eliminated 
altogether.  Project  10  may  thus  influence  the  Navy's  decision  on  training 
strategy,  but  is  ultimately  subject  to  the  chosen  strategy. 

b.  The  negative  results  of  a  generic  troubleshooting  training 
experiment  conducted  in  1983  for  BEE  students  (Technical  Report  NAVTRAEQUIPCEN 
82 -C-0119 :  Computer  Assisted  Instruction  System  Effectiveness  on 
Troubleshooting  Training.  August  1983)  suggests  that  context-free 
troubleshooting  training  must  be  done  very  early  in  the  training  track  to  be 
of  any  use.  (The  training  package  used  was  for  generic  troubleshooting 
training  and  is  generally  well-received  by  industry.  This  training  was 
administered  to  students  halfway  through  BEE  School.)  Context-specific 
troubleshooting  training  may  be  more  effective  at  later  points  in  the  training 
track. 
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PROJECT  9:  EMPIRICAL  EXAMINATION  OF  THE  MOST  COST-EFFECTIVE  AND  TRAINING- 
EFFECTIVE  MIX  OF  ACTUAL  EQUIPMENT  AND  TRAINING  SIMULATORS  FOR 
MAINTENANCE  TRAINING 

BACKGROUND.  While  it  is  known  that  effective  training  requires  a  nil  of 
actual  equipment  and  training  simulators,  little  empirical  work  has  been  done 
to  determine  the  best  mix.  Another  potential  question  focuses  on  the  most 
effective  sequence  in  which  the  equipment  is  to  be  used  in  a  training 
pipeline.  The  optimal  mix  is  influenced  by  the  type  of  tasks  involved.  To 
determine  training  time  allocation  among  actual-equipment  trainers  and 
simulators,  more  empirical  work  is  necessary. 

TASK  DESCRIPTION. 

a.  Review  research  findings  from  the  literature. 

b.  Conduct  systematic  experiments  to  assess  the  impacts  of  different 
mixes  of  actual  and  simulated  training  equipment. 

c.  Document  findings. 

NOTES . 

a.  To  our  knowledge,  no  work  in  this  area  is  planned  by  AFHRL. 

b.  ARI  is  conducting  work  in  this  area  with  outside  contractor  support 
(George  Mason  University). 
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PROJECT  8:  EMPIRICAL  EXAMINATION  OF  THE  RELATIONSHIPS  BETWEEN  TRAINING 

EQUIPMENT  DESIGN  FEATURES,  TRANSFER-OF -TRAINING ,  AND  TRAINING 
OBJECTIVES 

BACKGROUND.  Past  research  on  the  relationships  between  training  device 
"fidelity"  and  transfer-of-training  has  been  largely  academic  because  the 
interaction  with  training  objective  (type  of  tasks  trained)  was  ignored.  For 
example,  it  is  now  recognized  that  cognitive  training  (e.g.,  troubleshooting) 
benefits  from  low  physical  fidelity  though  high  functional  fidelity,  while 
manipulative  tasks  such  as  removing/replacing  assemblies  are  best  learned  on 
the  actual  equipment  or  a  close  replica  thereof  (whole  or  part-task  trainer). 
In  the  latter  case,  the  need  for  a  high  physical  fidelity  device  vice  actual 
equipment  is  justified  on  the  basis  of  cost  savings  and  the  fact  that  actual 
equipment  is  not  designed  for  student  abuse  so  that  instructors  are  reluctant 
to  use  it  for  hands-on  training  (instead,  the  hands-on  lab  work  advertised  in 
the  curriculum  is  more  a  hands-on  demonstration  conducted  by  the  instructor). 

A  third  dimension,  psychological  fidelity,  is  frequently  referred  to,  but  what 
it  is,  how  it  can  be  measured,  and  how  it  influences  transfer-of-training,  is 
not  clear.  Another  factor  not  very  well  covered  in  past  research  is  training 
strategy  --  a  factor  which  probably  has  the  largest  influence  on  the 
relationship  between  training  simulator  design  and  training  effectiveness. 
Training  strategy  determines  whether  the  training  objective  is  generic  versus 
equipment-specific  training,  whole  versus  part-task  training,  free  play  versus 
"aided"  training,  and  lock-step  versus  self-paced  training.  Finally,  little 
is  known  about  the  interactions,  if  any,  of  individual  abilities.  Much 
empirical  work  remains  to  be  done  to  assess  the  interactions  of  all  these 
factors  with  the  traditional  fidelity/training  effectiveness  relationship. 

TASK  DESCRIPTION. 

a.  Review  and  consolidate  previous  research  on  this  subject. 

b.  Conduct  systematic  experiments  to  assess  those  interactions  about 
which  insufficient  knowledge  is  available. 

c.  Document  results. 

NOTES . 


a.  ARI  is  planning  a  comprehensive  data  base  of  previous  research 
findings . 

b.  AFHRL  is  conducting  a  series  of  experiments  on  this  subject. 
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PROJECT  7:  DEVELOPMENT  OF  EXPANDED  TASK  ANALYSIS  PROCEDURES 

BACKGROUND.  There  is  a  growing  consensus  among  researchers  that  present  ISD 
procedures  are  not  suitable  for  deriving  training  device  specifications.  A 
different  procedure  must  be  developed  for  this  purpose,  but  there  is  no 
consensus  on  how  to  do  this. 

TASK  DESCRIPTION. 

a.  Review  the  existing  literature  on  this  topic  and  the  status  of 
research  in  the  other  Services. 

b.  Identify  the  specific  shortcomings  of  current  ISD  procedures. 

c.  Develop  revised  procedures  to  overcome  these  shortcomings. 

d.  Coordinate  with  the  other  Services  to  promulgate  a  tri-Service 
standard  for  expanded  task  analysis  procedures  required  to: 

(1)  Fill  the  voids  in  current  ISD  guides  and  procedures. 

(2)  Meet  the  specific  needs  for  training  device  specifications. 

NOTES . 

a.  ARI  is  planning  work  in  this  direction. 
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PROJECT  6:  TEST  AND  EVALUATION  OF  PROTOTYPE  FLEET  TRAINING  DELIVERY  SYSTEM 

BACKGROUND.  Following  development  and  approval  of  the  overall  concept, 
commercial  hardware  should  be  procured  and  software  developed  to  test  and 
evaluate  the  prototype  system. 

TASK  DESCRIPTION. 

a.  Procure  and  install  prototype  hardware  suitable  for  the  job  aiding 
software  (Project  3),  troubleshooting  tutorial  software  (Project  4),  and  other 
courseware  developed  in  accordance  with  the  plan  (Project  5). 

b.  Conduct  longitudinal  test  and  evaluation  of  the  system,  refining  or 
revising  components  of  the  system  accordingly. 

NOTES . 


a.  Unless  the  Navy  commits  itself  in  advance  to  conduct  this  type  of 
operational  testing  in  accordance  with  an  approved  plan  of  action  and 
milestones,  all  previous  recommended  research  projects  would  have  little 
practical  use  or  payoff. 
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PROJECT  5:  DEVELOPMENT  OF  A  FLEET  TRAINING  DELIVERY  SYSTEM  CONCEPT 

BACKGROUND.  The  Navy  currently  has  no  coordinated  plan  whether  and  how  to 
support  Fleet  technical  training  requirements.  ISD-based  training  course 
curricula  are  developed  with  the  knowledge  that  most  technical  training  is  not 
provided  at  Resident  Schools.  Yet,  development  of  on-board  training  packages 
has  not  occurred  in  the  past,  and  the  opportunities  for  OJT  are  limited. 

TYCOM  training  initiatives  are  strained  by  training  resources.  Identification 
of  skill  deficiencies  and  refresher  training  or  skill-enhancement  training 
requirements  is  currently  not  a  well-managed  process.  The  Navy  needs  an 
overall  concept  or  plan  for  meeting  these  Fleet  training  requirements 
(individual  maintenance  skills)  in  the  most  cost-effective  way. 

TASK  DESCRIPTION. 

a.  Proceed  with  NEPDIS  to  develop  a  comprehensive  data  base  for 
task/skill/knowledge  requirements. 

b.  Review  with  TYCOMs  the  current  Fleet  training  effort,  training 
requirements,  and  procedures  for  skill  appraisal  and  certification. 

c.  Identify  the  training  delivery  system  requirements: 

(1)  Student  load. 

(2)  Training  modules. 

(3)  Proficiency  testing. 

(Enhanced  job  aids  will  ensure  proficiency  only  for  frequently 
occurring  tasks). 

d.  Develop  a  cost-effective  concept  to  meet  this  need  using  available 
technology. 

NOTES . 

a.  The  training  strategy  espoused  by  this  study  eliminates  all  actual 
equipment  from  Resident  Schools,  replacing  it  with  technology  trainers  in  BEE 
School  (Project  11),  generalized  simulators  in  "A"  Schools  (Project  11),  and 
general-purpose  simulators  (such  as  the  GHTS)  and  part-task  trainers  in  "C" 
Schools.  TYCOM  training  activities,  under  the  recommended  strategy,  would 
have  operational  equipment  for  practical  job  training.  Thus,  this  project 
should  consider  the  availability  of  operational  equipment  (or,  "TTE") 
supplementing  the  training  delivery  system. 

b.  The  Navy  has  experience  with  various  media  suitable  for  this  system, 
except  for  interactive  video  disc  technology  in  which  ARI  has  invested 
considerable  efforts.  Technological  overkill  should  be  guarded  against, 
however;  the  GHTS  experience  shows  that  microfiche  can  be  applied  very  well  in 
troubleshooting  training.  Video  disc,  however,  is  needed  for  motion  dynamics. 
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i  PROJECT  4:  DEVELOPMENT  OF  EXPERT  TROUBLESHOOTING  TUTORIAL  SOFTWARE  (EQUIPMENT 

SPECIFIC) 

BACKGROUND.  Experience  has  shown  that  the  application  of  AI  principles  is 
effective  in  teaching  troubleshooting  skills.  What  is  currently  not  known  is 
;  the  best  mix  between  "feed-forward"  and  “feed-back"  in  teaching  these  skills. 

I  Nor  is  it  known  how  this  tutorial  information  should  be  modified  ("faded")  as 

a  function  of  the  student's  progress  or  how  to  mix  "product"  vice  "process" 
knowledge  of  results.  What  is  known  is  that,  in  the  instruction  mode,  the 
tutorial  information  roust  be  more  extensive  than  in  the  job  aiding  mode. 
Empirical  validation  is  necessary  to  resolve  these  issues  in  developing 
effective  tutorial  software. 

TASK  DESCRIPTION. 

a.  Develop  and  test  prototype  software  for  a  small  set  of  equipments 
and  systems  of  graded  complexity.  Assess  the  impacts  of  systematic  changes 
in:  the  mix  of  feed-back  and  feed-forward  information;  the  extent  of  fading; 

'  and  the  type  of  information  (process  vs.  result). 

b.  Analyze  and  document  empirical  results. 

c.  Revise  the  draft  specification  for  tutorial  troubleshooting  software 

I  (Project  3)  accordingly. 

REFERENCE. 

Rasmussen,  Jens,  and  Rouse,  William  B.  (eds).  "Human  Detection  and  Diagnosis 
of  System  Failures,"  Proceedings  of  NATO  Symposium.  August  1980.  New 
l  York:  Plenum  Press,  1981. 

l 

NOTES . 


a.  The  Navy  already  has  a  system,  the  Generalized  Maintenance  Training 
Simulator  (GMTS),  with  equipment-specific  software  for  the  AN/WSC-3, 

AN/SPA-66,  and  Fleet  Communications  System.  This  system  could  be  used  in 
support  of  Project  4  instead  of  starting  from  scratch.  One  question  that  must 
be  considered  is  whether  current  GMTS  software  faithfully  reflects  the 
uncertainties  of  test  measurement  validity  encountered  in  the  real  world 
(i.e.,  marginal  values,  neither  yes  nor  no).  ARI's  ACTS,  reportedly,  is 
designed  to  handle  that  aspect  of  diagnostic  activity  very  well. 
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PROJECT  3:  DEVELOPMENT  OF  SPECIFICATIONS  AND  GUIDES  FOR  JOB  AIDING  SOFTWARE 

BACKGROUND.  The  previous  two  projects  are  not  recommended  for  their  own  sake, 
but  as  prerequisite  efforts  in  developing  effective  job  aiding  software, 
tutorial  enhancements  to  that  software  in  order  to  transition  novices  to 
experts  as  soon  as  possible,  and  effective  troubleshooting  training  software. 
To  institutionalize  this  approach,  guidelines  and  specifications  are  necessary 
in  support  of  acquisition  of  this  software.  Such  data  are  currently  not 
available. 

TASK  DESCRIPTION. 

a.  Following  completion  of  Projects  1  and  2,  develop  draft 
specifications  and  guidelines  for  job  aiding  software  and  tutorial  software 
for  use  in  job  aids  and  maintenance  training  curricula. 

b.  Validate  the  specifications  through  pilot  applications  (see  Project 

4). 

c.  Coordinate  with  the  other  Services  and  promulgate  a  military 
specification  for  use  in  contracts. 


NAVTRAEQUIPCEN  MDA903-81-C-0166-1 


NOTES . 

a.  ONR  and  DARPA  have  sponsored  considerable  research  in  Artificial 
Intelligence  (AI)  and  novice  versus  expert  behavior  in  problem  solving  but,  it 
appears  that  this  project  does  not  duplicate  anything  done  in  the  past. 

b.  NAVPERSRANDCEN  has  embarked  on  a  research  program  to  utilize  AI 
concepts  for  presenting  technical  information  to  technicians  but  a  project 
like  the  one  described  here  is  not  included  in  that  program. 

c.  Current  Al-based  training  systems,  such  as  ARI's  Adaptive  Computer 
Training  System  (ACTS),  employ  a  theoretical  model  of  expert  performance  based 
on  maximizing  "expected  utility"  from  each  troubleshooting  step.  There  is 
some  question  as  to  how  realistic  that  concept  is. 

d.  AFHRL  has  no  known  research  planned  in  this  area. 

e.  Past  research  in  this  area,  according  to  ARI,  suggests  the  following 

(1)  Technicians  tend  to  have  difficulty  verbalizing  why  they  are 
doing  what  they  are  doing;  it  may  be  necessary  to  add  video  and  query  the 
technician  after  each  session. 

(2)  The  first  step,  prior  to  experimentation,  should  include  a 
detailed  analysis  of  existing  technical  documentation  to  develop  a  conceptual 
model  (cognitive  and  psychomotor  demands)  of  the  target  system. 

(3)  The  on-line  protocols  developed  in  each  session  can  be  used  for 
improving  this  conceptual  model  and  ultimately  for  task  analysis  validation 
and  training. 
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PROJECT  2:  ANALYSIS  OF  EXPERT  TROUBLESHOOTING  STRATEGIES  AND  BEHAVIOR 

BACKGROUND.  Past  research  has  indicated  that  experts  rely  on  a  mental  model 
of  the  functional  interrelationships  within  a  system;  that  they  possess 
specific  cognitive  abilities  (e.g.,  problem  sensitivity,  information  ordering, 
flexibility  of  closure,  spatial  orientation,  visualization,  selective 
attention,  and  attention  to  detail)  which  non-experts  may  not  possess;  that 
they  utilize  the  information  gained  from  test  measurements  much  better  than 
non-experts  do,  especially  measurements  that  identify  sections  of  the 
equipment  as  "good";  and  that  they  are  able  to  utilize  built-in  testability 
features  more  productively  than  non-experts  do.  More  specific  details  on  what 
makes  an  expert  need  to  be  learned  to  apply  this  knowledge  for  expediting  the 
transition  from  non-expert  to  expert  troubleshooting  performance. 

TASK  DESCRIPTION. 

a.  Interview  subjects  (experts  and  novices)  to  assess  their  knowledge 
of  selected  systems  and  the  procedures  they  profess  to  follow  in  diagnosing 
failures.  Tape  record  each  interview. 

b.  Conduct  a  series  of  experiments,  observing  how  the  experts 
troubleshoot  and  isolate  system  failures.  Encourage  subjects  to  think  aloud, 
articulating  why  they  do  what  they  do.  Tape  record  each  session. 

c.  The  number  of  experiments  (or  the  size  of  the  experiment  if  all 
variables  are  manipulated  in  one  experiment)  depends  on  the  scope  and  validity 
desired: 

(1)  Individual  differences  of  expert  technician  subjects. 

(2)  Different  types  of  equipment. 

d.  Conduct  identical  experiments  with  the  control  group  of  novice 
troubleshooters . 

e.  Anchor  the  expert  performance  in  each  experiment  by  means  of  a 
testability  analysis  model  (LOGMOD)  identifying  the  minimum  steps  required  for 
fault  isolation. 

f.  Analyze  and  document  results,  identifying: 

(1)  How  good  the  experts  are. 

(2)  What  specific  factors  explain  their  performance  (cognitive 
abilities,  experience,  knowledge). 

(3)  What  the  key  differences  are  between  experts  and  novices. 
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b.  While  current  R&D  programs  (PEAK,  NTIPP,  Hybrid  JPAs)  are  designed 
to  address  this  issue  in  some  form,  a  more  rigorous  examination  is  deemed 
necessary  to  achieve  the  full  potential  of  aiding  technology. 

c.  It  appears  that,  neither  ARI  nor  AFHRL  have  a  project  of  this  type 
in  their  research  programs. 
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APPENDIX  A 

CANDIDATE  RESEARCH  AND  DEVELOPMENT  PROJECTS 


PROJECT  1:  ANALYSIS  OF  TECHNICAL  INFORMATION  NEEDS  IN  MAINTENANCE  TASK 

PERFORMANCE 

BACKGROUND.  The  effectiveness  of  a  job  aid  is  determined  by  many  factors. 

One  factor  is  the  extent  to  which  the  technical  information  provided  by  the 
aid  matches  the  information  needs  of  the  technician  in  terms  of  content, 
format,  detail,  sequence,  chunk  (amount  of  information  per  frame),  and  type 
(fact,  concept,  rule,  procedure,  or  principle).  There  is  only  limited 
knowledge  about  these  specific  needs  and,  especially,  how  they  vary  with 
different  conditions  (equipment,  type  of  task,  operational  environment, 
experience  level,  individual  differences).  This  void  in  knowledge  must  be 
eliminated. 

TASK  DESCRIPTION. 

a.  Conduct  a  series  of  experiments  observing  subjects  performing 
maintenance.  Subjects  have  no  Technical  Manuals  (TMs),  tools,  or  test 
equipment.  Experts  are  present  to  respond  to  questions  by  subjects,  including 
requests  for  specific  information,  tools,  and  test  equipment.  Observers 
document  what  goes  on,  using  tape  recorders. 

b.  The  number  of  experiments  (or  the  size  of  the  experiment  if  all 
variables  are  manipulated  in  the  same  experiment)  is  determined  by  the 
following  factors: 

(1)  Individual  differences  (contrasting  cognitive  abilities  of 

subjects) . 

(2)  Different  equipment  (different  levels  of  complexity). 

(3)  Task  coverage. 

(4)  Maintenance  levels  ("0"  and  "I"). 

(5)  Experience  levels  (e.g.,  less  than  one  year,  two  years,  four 
years).  (To  increase  the  insights  gained  from  the  project,  the  same 
experiments  could  be  conducted  on  equipment  other  than  combat  systems). 

c.  Analyze  and  document  the  specific  information  needs  identified  as  a 
function  of  experience  level  and  individual  differences. 

NOTES . 

a.  The  limited  success  of  JPAs  is  attributed  to  the  lack  of  knowledge 
this  project  is  intended  to  provide. 
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TD 

- 

Training  Device  Technician 

TH 

- 

technical  manual  or  Torpedoman's  Hate 

TRADOC 

- 

U.S.  Training  and  Doctrine  Command 

TTE 

- 

technical  training  equipment 

TYCOM 

- 

Type  Commander 

UHF 

• 

ultra-high  frequency 

UNITREP 

- 

unit  status  and  identify  report 

U.S. 

- 

United  States 

USDRE 

- 

Under  Secretary  of  Defense  for  Research  And  Engineering 

USN 

- 

United  States  Navy 

VADM 

Vice  Admiral 

VHSIC 

- 

very  high-speed  integrated  circuit 

VLSI 

- 

very  large-scale  integration 

VLSL 

- 

very  large-scale  logic 

VRA 

- 

weapon  replaceable  assembly 

2-M 

miniature/microminiature 

3-M 

- 

Maintenance  and  Material  Management 

I 
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(6)  Advanced  communications  maintenance  trainer. 

(7)  Sonar  maintenance  trainer. 

REFERENCES. 

Parker,  E.  L.  Generalized  Training  Devices  for  Avionic  Systems  Maintenance. 
Santa  Barbara,  California:  Anacapa  Sciences,  August  1974. 

Parker,  E.  L.  Applications  and  Design  Characteristics  of  Generalized  Training 
Devices .  Santa  Barbara,  California:  Anacapa  Sciences,  March  1971. 
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PROJECT  12:  CONDUCT  OF  COST-EFFECTIVENESS  STUDIES  OF  MAINTENANCE  TRAINING 
EQUIPMENT 

BACKGROUND.  The  lack,  of  adequate  cost-effectiveness  studies  of  maintenance 
training  simulators  has  been  pointed  out  by  several  observers.  While  studies 
confirm  that  simulators  are  cheaper  than  actual  equipment  and  students  trained 
on  simulators  perform  equally  as  well  as  those  trained  on  actual  equipment, 
the  limitations  of  such  studies  are  obvious.  To  demonstrate  convincingly  the 
cost-effectiveness  of  training  simulators,  longitudinal  studies  must  be 
performed,  taking  into  account  a  number  of  other  aspects  as  well;  e.g.,  mix  of 

equipment  used  in  the  training  pipeline,  training  strategy,  skill  retention. 

These  studies  could  be  combined  with  some  previously  referred  to  (Projects  8 
and  9)  in  assessing  training-effectiveness  of  alternative  training  strategies. 

TASK  DESCRIPTION. 

a.  Consolidate  the  findings  from  past  evaluation  studies. 

b.  Identify  the  shortcomings  of  these  evaluations. 

c.  Develop  a  comprehensive  evaluation  plan. 

d.  Conduct  experimental  studies. 

e.  Document  findings. 

REFERENCES. 

Orlansky,  J.,  and  String,  J.  Cost-Effectiveness  of  Maintenance  Simulators  for 
Military  Training.  Arlington,  Virginia:  Institute  for  Defense  Analyses, 
August  1981. 

NOTES . 

a.  AFHRL  is  conducting  work  in  this  area,  but  the  results  will  be 
subject  to  similar  limitations  as  those  of  past  studies  (see:  Louis  F. 
Cicchinelli  et  al..  Relative  Cost  and  Training  Effectiveness  of  the  6833 
Converter/Flight  Control  System  Simulators  As  Compared  To  Actual  Equipment 
(Brooks  Air  Force  Base,  Texas:  Air  Force  Human  Resources  Laboratory,  December 
1982)). 

b.  ARI  is  planning  to  conduct  studies  in  this  area  under  the  AMTESS 
program. 

c.  The  Navy's  current  evaluation  of  the  EEMT  is  subject  to  the  same 
limitations  as  indicated  for  AFHRL' s  work. 

d.  According  to  some  observers,  the  utility  of  this  project  may  be 
limited  because  the  rate  of  change  of  technology  is  too  rapid  to  obtain 
meaningful  results. 
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PROJECT  13:  DEVELOPMENT  OF  TRAINING  EQUIPMENT  ACQUISITION  MANUAL  AND  POLICY 

BACKGROUND.  The  Navy  lacks  a  standard,  uniform  procedure  for  the  acquisition 
of  training  equipment.  There  is  no  effective  review  process  to  determine  if 
training  requirements  resulting  from  a  new  weapon  system  acquisition  program 
could  be  met  by  training  equipment  in  the  existing  inventory.  Rather,  it  is 
assumed  that  every  new  prime  system  requires  one  or  more  new  trainers. 
Additionally,  the  potential  suitability  of  a  new  trainer  to  meet  the  training 
requirements  of  more  than  one  new  system  is  not  considered.  Furthermore,  the 
trainer  design  characteristics  are  more  influenced  by  contractor  proposals 
than  by  a  rigorous,  independent  assessment  of  the  functional  requirements  for 
a  new  trainer.  To  improve  the  situation  and  achieve  more  training 
effectiveness  from  training  device  investments,  current  policies  must  be 
tightened  in  one  respect  (procedure  and  process)  and  loosened  in  another 
(planned  test  and  evaluation  of  first-article  trainers  treat  them  as  prime 
equipment  with  much  unnecessary  detail  and  administrative  burden).  More 
importantly,  prime  equipment  program  (project)  managers  must  be  given  the 
decision  aids  they  are  currently  lacking. 

TASK  DESCRIPTION. 

a.  Expand  the  training  device  data  base  to  include  data  elements 
permitting  retrieval  of  device  information  keyed  to  a  taxonomy  of  devices  by 
type,  tasks  simulated,  and  utilization. 

b.  Consolidate  empirical  and  case  history  information  on  current 
devices  into  a  training  device  design  guidelines  manual. 

c.  Consolidate  the  results  of  previous  R&D  (Projects  7-11)  and  the 
above  tasks  into  a  Training  Equipment  Acquisition  Manual  in  a  format  suitable 
for  addressing  the  key  issues  in  acquisition: 

(1)  Validation  of  the  need  for  a  new  trainer. 

(2)  Identification  of  the  mix  of  trainer  types  required. 

(3)  Specification  of  the  functional  characteristics  required. 


NOTES . 

a.  ARI  is  embarked  on  a  five-year  program  that  is  ultimately  to  result 
in  a  decision  support  system  for  training  equipment  requisition  decisions. 
This  R&D  program  appears  to  be  well-designed  and  is  very  comprehensive. 

b.  AFHRL's  past  efforts  in  this  area  have  resulted  in  preliminary 
manuals;  see,  for  example,  AFHRL-TP-81-51 ,  Maintenance  Training  Simulator 
Design  and  Acquisition:  Handbook  of  ISP  Procedures  for  Design  and 
Documentation.  February  1982.  AFHRL  plans  to  complete  its  work  in  this  area 
by  the  end  of  the  current  fiscal  year  with  issuance  of  a  set  of  introductory 
handbooks  for  ISD  teams  and  acquisition  managers. 
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PROJECT  14:  DEVELOPMENT  OF  FLEET  PROJECT  TEAM  GUIDE 

BACKGROUND.  Responsibilities  of  the  Fleet  Project  Team  (FPT)  are  specified  by 
OPNAV  Instruction  1551. 7B  (April  1977).  The  FPT  acts  as  the  user 
representative  and  subject  matter  expert  in  training  device  acquisition.  Its 
specific  duties  include: 

a.  Provide  guidance  to  the  developer. 

b.  Develop  training  syllabus  in  which  the  device  will  be  used. 

c.  Assist  developer  in  specifying  acceptance  criteria. 

d.  Identifying  trainer  deficiencies  requiring  correction  before  ready- 
for-training  (RFT)  approval. 

The  FPT  guide  spells  out,  in  more  detail,  the  functions  of  the  FPT  and  the 
milestones  or  activities  associated  with  device  acquisition.  It  also  makes 
clear  that  the  FPT  role  is  advisory  only,  subject  to  invitation  or  request  to 
comment  on  the  various  acquisition  requirements  (military  characteristics, 
design  reviews,  ILS  plans,  inspections  and  tests)  by  the  developer 
(NAVTRAEQUIPCEN)  who  has  technical  and  contractual  authority.  In  view  of 
personnel  turnover  and  the  lengthy  acquisition  cycle  of  complex  trainers,  the 
effectiveness  of  the  FPT  could  be  increased  by  providing  them  with  more 
detailed  aids.  One  area  in  which  the  FPT  can  be  instrumental  is  to  promote 
user  acceptance,  but  that  area  is  not  addressed  in  the  current  FPT  guide.  In 
all  other  areas,  the  guide  tells  what  is  to  be  done,  not  how  it  should  be  done 

TASK  DESCRIPTION. 

a.  Consolidate  findings  from  previous  studies  on  the  issue  of  user 
acceptance. 

b.  Develop  a  School  Introduction  Guide  outlining  how  user  acceptance 
can  be  gained  through  early  interface  with  instructor  personnel,  proper 
introduction  of  the  device,  and  timely  availability  of  the  training  syllabus. 

c.  Develop  a  Training  Device  Evaluation  Guide. 

REFERENCES . 

Mackie,  Robert  R.,  et  al.,  Factors  Leading  to  the  Acceptance  or  Rejection  of 
Training  Devices.  Goleta,  California:  Human  Factors  Research,  Inc., 
August  1972. 
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PROJECT  15:  STANDARDIZATION  OF  INSTRUCTOR  AND  STUDENT  STATIONS 

BACKGROUND.  The  benefits  of  modular  design  and  standardization  of  common 
modules  have  been  demonstrated  in  many  cases  and  represent  the  chief  lesson 
learned  from  past  experience  with  ATE  in  the  Department  of  Defense.  The  same 
concept  can  be  applied  to  maintenance  training  simulators  because  they  have 
some  modules  in  common,  regardless  of  application:  the  instructor  station  and 
student  station.  Significant  acquisition  cost  savings  can  be  achieved  by 
developing  standards  for  these  modules  and  prescribing  these  standards  for  new 
maintenance  trainers,  whenever  feasible,  so  that  they  do  not  have  to  be 
redesigned  for  every  peculiar  maintenance  trainer.  Savings  in  support  costs 
will  also  be  significant  as  a  result  of  standard  spares  and  repair  parts, 
lower  stockage  costs,  and  standard  manuals. 

TASK  DESCRIPTION. 

a.  Develop  a  detailed  taxonomy  of  maintenance  training  equipment, 
including  types  and  attributes. 

b.  Conduct  a  commonality  analysis. 

c.  Identify  the  functional  requirements  of  instructor  and  student 
stations . 

d.  Develop  specifications  and  design  guidelines. 

e.  Coordinate  with  the  other  Services  to  promulgate  a  military  standard. 
NOTES . 

a.  The  Air  Force  program  office  for  simulators  (SIHSPO)  has  this 
subject  in  its  program  and  is  supposed  to  take  the  lead  in  this  effort. 
Coordination  with  SIHSPO  would  be  well-advised. 
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APPENDIX  B 

DEFENSE  SCIENCE  BOARD  1982  SUMMER  STUD? 
ON  TRAINING  AND  TRAINING  TECHNOLOGY 


INTRODUCTION 

The  Defense  Science  Board  (DSB)  1982  Summer  Study  addressed  for  the  third 
time  in  seven  years  the  topic  of  training  and  training  technology.  Reasons 
advanced  for  this  renewed  focus  were  the  lack  of  action  by  the  Department  of 
Defense  (DoD)  on  previous  training  recommendations  (1976  DSB),  the  shrinking 
manpower  pool,  limited  supply  of  high  mental  category  accessions,  rapid 
turnover  of  personnel,  increasing  technological  complexity  of  equipment,  and 
increasing  ratios  of  equipment  to  personnel.  The  training  and  training 
technology  panel  was  chaired  by  Admiral  Isaac  C.  Kidd,  Jr.,  USN  (Ret.)  and 
comprised  four  subpanels:  organization  and  acquisition  of  training,  training 
technology,  manpower  requirements  and  skill  development,  and  operational 
training. 

The  panel's  findings,  conclusions,  and  recommendations  are  summarized  in 
this  appendix  for  reference  purposes.  This  summary  is  based  on  the  panel's 
final  briefing  charts  as  distributed  by  the  Under  Secretary  of  Defense  for 
Research  and  Engineering  (USDRE). 

FRAME  OF  REFERENCE 

The  panel  focused  its  effort  on  the  following  key  questions: 

a.  How  effective  is  current  training? 

b.  How  is  training  effectiveness  measured? 

c.  What  are  the  cost-benefit  trade-offs  between  simulators  and  other 
training  aids  versus  actual  equipment  for  training? 

d.  What  technologies  exist  that  would  improve  the  training  of  operator 
and  maintenance  personnel? 

e.  How  much  improvement  is  possible  and  what  areas  of  training  could 
benefit  most  by  adopting  new  approaches? 

f.  What  actions  are  necessary  to  improve  implementation  and  utilization 
of  advances  in  training  technology? 

With  respect  to  the  first  four  questions,  the  panel  found  that  current 
training  is  marginally  effective;  training  effectiveness  is  measured 
subjectively  and  poorly;  cost  data  and  performance  data  necessary  for 
cost-benefit  trade-offs  are  very  limited  (but  in  the  few  cases  they  are 
available,  simulators  cost  20  to  60  percent  less  to  acquire  than  actual 
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equipment,  and  are  equally  effective  as  measured  by  student  achievement);  new 
technologies  applicable  to  training  are  available  and  include  computer-based 
simulation,  stimulation,  communication,  data  collection,  reduction,  and 
retrieval  as  well  as  new  means  of  computer  input  and  output. 

With  regard  to  the  fifth  question,  the  panel  assessed  present  equipment 
and  personnel  performance  at  40  percent  of  optimal  potential  and  estimated 
this  could  be  improved  to  70  percent  with  proper  application  of  training 
technology  and  planned  reliability/maintainability  improvements  in  weapon 
systems.  The  training  areas  to  benefit  most  from  new  technology  were  found  to 
be  individual  advanced  skill  training  (both  operators  and  maintainors)  and 
team  or  unit  training.  The  sixth  question  was  answered  negatively;  the  panel 
found  a  continued  lack  of  coordination  between  "people  people,”  "hardware 
people,"  and  "training  people"  due  to  a  lack  of  guidance  from  the  top. 
Addressing  the  final  question,  the  panel  estimated  that  expenditures  for 
training  have  fallen  out  of  balance  with  those  for  prime  equipment.  While 
present  training  is  expensive  (individual  training  at  service  schools  costs 
$12.8  billion  a  year  with  20  percent  of  military  personnel  involved  in 
training  as  students,  instructors  or  school  support;  the  additional  costs 
involved  in  unit  training  and  advanced  training  not  in  schools  is 
unquantif iable) ,  the  panel  found  that  increased  spending  for  high-technology 
training  methods  and  equipment  holds  the  promise  of  quick  return  on  investment 
by  improving  personnel  ability  to  use  and  maintain  military  equipment.  It 
estimated  that  an  additional  $2.6  billion  a  year  could  be  used  effectively  for 
improved  training  technology,  including  devices,  software,  and  courseware. 

CONCLUSIONS 

The  panel' 8  conclusions,  organized  by  subpanel,  are  summarized  in  Table 
B-l .  Not  included  are  those  conclusions  pertaining  to  operational  and  unit 
training  since  this  study  is  focused  on  individual  maintenance  skills. 

ISSUES  AND  RECOMMENDATIONS 

The  panel '8  recommendations  were  briefed  by  a  subpanel.  Only  those 
issues  and  associated  recommendations  applicable  to  maintenance  training  are 
listed  here,  quoted  verbatim  from  the  briefing  report. 

MANPOWER  REQUIREMENTS  AND  SKILL  TRAINING 

ISSUE  #1:  Demographic  projections  and  "technology  creep"  indicate 
manpower  problems. 

Recommendations: 

a.  Use  technology  to  reduce  failure  rate  and  simplify  operator/ 
maintainer  tasks  (Project  Office,  2X  of  system  FSD  cost). 

b.  Explore  self-motivating  arcade-like  devices  to  increase  performance 
level  of  recruits  (R&D  agencies,  three  years  at  $250K/system/year) . 
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c.  Require  use  of  contemporary  methodology  such  as  HARDMAN  to  match 
hardware  to  people  (Project  Office,  $3M/system). 

t 

ISSUE  #2:  Educational  deficits  can  result  in  serious  under-utilization  of 
the  recruit's  talent. 


Recommendations : 

a.  Develop  technology  to  match  instruction  to  ways  recruits  learn  best 
(Service  Schools,  five  years  at  $5M/year). 

b.  Use  innovative  ways  to  provide  necessary  English  language  shills 
(Service  Schools,  $5M/year). 

c.  Use  technology  such  as  video  storage  in  school  and  field  to  teach 
students  about  equipment  they  will  be  using  (R&D  agencies,  five  years  at 
$10M/year ) . 

d.  Use  transportable  devices  in  the  field  to  broaden  understanding  for 
career  growth  at  leadership  (R&D  agencies,  five  years  at  $5M/year). 

ISSUE  #3:  Benefit  of  CAI  and  new  technology  for  field  refresher  and  other 
training  are  limited  by  slow  introduction  into  the  training 
base. 

Recommendations: 


a.  Innovative  ways  to  make  trainers  accept  technology  changes  such  as 
CAI  and  learn  to  use  them  (Chiefs  of  Staff,  cost  to  be  included  in  demo 
projects) . 

b.  Accelerate  introduction  of  CAI  into  the  "schoolhouse"  to  allow 
transportability  of  this  training  to  the  field  (Service  Schools,  five  years  at 
t46M/year ) . 


c.  Build  CAI  into  training  package  of  all  new  operational  systems 
(Project  Office,  0.5T-10X  of  system  cost). 


ISSUE  //4:  Adequate  data  not  yet  assembled  to  determine  cost-effectiveness 
of  training  methods  and  devices. 


Recommendations : 

a.  Undertake  demonstration  projects  and  provide  high  technology 
testbeds  for  training  and  performance  measurement  (R&D  agencies,  five  years  at 
$16H/year ) . 

b.  Direct  where  possible  embedded  training  and  performance  measurement 
be  built  into  new  systems  (Material  Command,  $1M  or  21  of  system  cost). 
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c.  Develop  and  adopt  quantitative  performance  measures  (R&D  agencies, 
Training  Commands,  five  years  at  $12M/year). 

d.  Establish  a  repository  for  all  training  data  (OSD  Field  Agency,  five 
years  at  $250K/year). 


TRAINING  TECHNOLOGY 

ISSUE  #1:  Advanced  software  techniques  exist  which  promise  gains  in 

software/courseware  production  efficiency,  but  they  are  not  yet 
applied. 

Recommendat ions: 

(Short  Term) 

a.  Encourage  use  of  common  courseware  modules  and  "user  friendly" 
interfaces  (AI  based). 

b.  Direct  future  CAI  acquisitions  to  specify  transportable  software 
including  operating  system  (Joint  Services,  $  negligible). 

(Long  Term) 

a.  Direct  Machine  Intelligence  R&D  efforts  in  automatic  programming, 
information  extraction,  expert  systems,  and  "good  teacher"  models  reducing 
courseware  production  cost  (DoD  agencies,  I12M/FY84-89) . 

ISSUE  #2:  Weapon  systems,  based  on  digital  technology,  can  be  used  to 
provide  more  effective  training  and  performance  measurement. 

Recommendations: 

a.  Embedded  training:  new  weapons  systems  should  include  means  of 
providing  simulated  targets  and  environmental  conditions  (DSARC  &  Material 
Development,  10T  of  weapon  system  development). 

b.  Develop  and  incorporate  performance  measurement  capabilities  in  new 
weapons  systems  (Material  Development  agencies  and  R&D  labs,  $5M/year  for 
three  years  to  develop  methodology;  negligible  unit  costs  for  implementation). 

ISSUE  #3:  Satellite  communications  capacity  exists  which  may  be  used  for 
remote  training,  maintenance,  technical  manual  updating,  and 
maintenance  teleconferencing. 

Recommendations: 

a.  Develop  cost-effective  ground  stations  with  hardcopy  and  video 
recording  capability  (Joint  Services,  $20H). 
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b.  Pursue  technologies  related  to  data  compression  (USDHE,  $5H). 

c.  Establish  a  group  from  the  training  commands  to  explore  satellite 
capability  (USDRE,  $  negligible). 

ISSUE  #4:  Critical  training  technologies  need  additional  emphasis  and 
focus  by  USDRE. 

Recommendations: 

a.  USDRE  assign  responsibilities  within  DARPA  and  Services  for  emphasis 
on  key  technologies  —  such  as: 

(1)  Voice  recognition  and  synthesis. 

(2)  Voice  forwarding  and  speech  storage. 

(3)  Interactive  display  technology  (soft  and  hard) 

(4)  Personal  microprocessor  training  aids. 

b.  Use  advanced  training  devices  as  testbed  for  application  of  VHSIC. 

c.  Establish  "brainstorming"  sessions  with  industry  to  develop  new 
ideas . 


ORGANIZATION  AND  ACQUISITION 

ISSUE  #1:  The  need  for  proper  training  management  and  training  technology 
mandates  that  OSD  establish  the  proper  environment  to  support 
the  training  initiatives  of  the  Services  and  JCS. 

Recommendation: 

a.  Establish  an  OSD  steering  committee  for  training  matters  (SECDEF,  $ 
negligible) . 

ISSUE  #2:  A  single  proponent  within  each  Service  is  required  to  ensure 

proper  emphasis  and  consideration  of  new  training  technologies 
and/or  devices. 


Recommendation: 

a.  SECDEF  direction  to  the  Services  to  provide  a  single  proponent 
within  each  Service  for  consideration  of  new  training  technologies  and  devices 
(SECDEF,  Service  Heads,  $  negligible). 
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ISSUE  #3:  The  history,  tradition,  and  organization  of  Service  labs  often 
appears  to  establish  research  priorities.  An  eiample  is  the 
minimal  emphasis  on  training  technology. 

Recommendation: 

a.  Direct  the  Services  through  their  directors  of  laboratories  to 
formulate  a  program  to  increase  the  priority  for  training  technology  (USDRE, 
allocate  within  current  $s). 

ISSUE  #4:  The  time  required  for  weapon  system  data  to  stabilize  in  the 
development  process  often  tends  to  cause  training  device 
development  to  start  late. 

Recommendations : 

a.  Initiate  training  device  development  earlier  in  the  weapon  system 
acquisition  cycle  (USDRE/Service  Heads). 

b.  Consider  providing  relief  in  acquisition  process  for  training 
devices  (USDRE,  Service  Heads,  can  decrease  annual  DoD  training  costs  by  51) . 

SUMMARY 

The  recommendations  of  the  DSB,  as  summarized  in  its  Final  Report,  are 
shown  in  Table  B-2. 
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TABLE  B- 1 .  SUMMARY  OF  DSB  CONCLUSIONS 


MANPOWER  REQUIREMENTS  AND  SKILL  TRAINING 

-  Recruit  quality  and  quantity  goals  met  in  F781;  FY82  will  be 
even  better,  but... 

-  Future  shortages  may  be  anticipated 

—  declining  manpower  pool,  changing  demographic  mix,  expand¬ 
ing  economy 

--  increasing  skill  requirements. 

-  Training  technology  offers  significant  and  exciting  solutions 
to  some  of  the  challenges. 

-  More  and  better  data  are  needed  to  guide  future  applications. 

TRAINING  TECHNOLOGY 

-  Microprocessor  based,  interactive  video  disk  systems  have 
revolutionized  the  instructional  industry. 

-  DoD  need  not  fund  the  entire  technology,  but  only  its  special 
needs . 

-  Software,  including  courseware  development,  is  the  dominant 
cost  factor  for  computer-based  instructional  systems. 

-  Potential  exists  for  improved  training  and  performance  measure¬ 
ment  using  embedded  simulation  and  stimulation. 

-  New  technologies  such  as  VHSIC,  advanced  storage  techniques, 
voice  synthesis  and  recognition  will  produce  improvements. 

ORGANIZATION  AND  ACQUISITION 

-  OSD  not  tuned  to  Services  training  management  and  training 
technology  needs;  nor  are  the  Joint  Chiefs. 

-  Control  and  management  of  6. 1-6.4  training  technology  funds  are 
fragmented  within  most  of  the  Services.  This  fragmentation 
causes  inadequate  emphasis  and/or  acceptance  of  new  training 
technologies . 

-  Currently  available  data  on  individual/collective  performance 
is  not  sufficient  to  support  effective  management  of  training 
resources  (system  and  non-system) . 

-  Training  devices  often  seriously  lag  the  introduction  of  the 
weapon  system. 

-  Service  laboratories  direct  very  little  effort  to  the  potential 
improvements  in  training  through  the  application  of  con¬ 
temporary  technologies. 


Excludes  conclusions  on  operational  training  and  unit  training 
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TABLE  B-2.  SUMMARY  OF  DSB  RECOMMENDATIONS 


Ares1 

1. 

2. 

3. 

A  4. 


5. 

6. 

1. 

2. 

3. 

4. 

5. 

6. 

1. 


Specific  Actions  Required 


Establish  an  OSD  Steering  Committee  for  Training  and  Training  Tech¬ 
nology.  Focus  is  to  be  on  policy  review  and  coordination  of  initiatives 
to  produce  sore  effective  training  through  use  of  existing  and  new 
technology. 

Establish  a  Defense  Training  Data  and  Analysis  Center  for  all 
training-related  data. 

Revise  acquisition  process  to:  (a)  ease  procurement  specifications  and 
standards  commensurate  with  training  device  use  and  (b)  acquire  training 
requirements  data  earlier  in  weapons  system  development  cycle. 

Increase  use  of  analytical  methoda  to:  (a)  assess/project  impact  of 
manpower  skills/performance  of  recruits  to  meet  system  needs,  do  not 
pool  on  new  weapons  systems  and  (b)  identify  where  training  may  increase 
skills/performance  of  recruits  to  meet  system  needs,  do  not  wait  for 
more  analysis/assessment.  There  are  enough  data  to  proceed  now. 

Direct  the  Military  Departments  to  increase  funding  and  management 
emphasis  on  research  and  development  of  training  technology,  its  appli¬ 
cation  and  its  payoff. 

Direct  the  Military  Departments  to  provide  a  single  point  of  contact  for 
proponency  and  coordination  of  training  and  training  technology. 


Provide  the  Reserve  Components  with  up-to-date  training  technology  and 
equipment.  Support  training  to  meet  the  unique  needs  of  the  Reserve 
Component  training  objectives,  schedules,  and  environaient. 

Support  the  funding  of  research,  development  and  applications  of  tech¬ 
nologies  for  unit  training. 

Support  research,  development  and  use  of  war  games  that  provide  intelli¬ 
gent  adversaries  and  realistic  conditions  to  promote  effective  combat 
leadership  training. 

Upgrade  ranges;  increase  number/size  of  ranges  baaed  on  requirements  for 
operational  training  and  testing  of  current/prograamed  weapons. 

Accelerate  use  of  computer-based  instructional  methods  (including  CAI 
and  CMI)  in  the  schoolbouse  and  on  the  job  via  portable  aids  and/or 
embedded  training  systems. 

Use  transportable  devices  in  the  field  to  broaden  understanding  and 
general-skill  knowledge  for  career  growth  and  leadership. 


Establish  (a)  a  research  and  development  program  on  performance  measures 
to  develop  criteria,  methodology  and  equipment  for  use  at  all  revels  of 
training  and  (b)  demonstration  projects  for  new  training  technology  to 
collect  data  on  performance  and  cost  effectiveness. 

Increase  exploration  and  uae  of  current/advanced  technology  devices 
(e.g.,  arcade-like  games)  to  motivate  and  teach  functional  skills. 

Increase  support/funding  for  research,  development  and  use  of  the 
following  technologies;  voice  recognition,  interactive  display,  personal 
aids  and  VHSIC. 


4.  Develop  and  incorporate  embedded  training  and  performance  measurement/ 
recording  capsbilities  for  new  weapon  and  support  systems. 

5.  Direct  future  acquisition  of  training  equipment  to  use  transportable 
software  and  to  be  "user  friendly”  in  meeting  instructional  needs. 


*A  a  Organization  and  Management;  B  =  Planning  and  Application;  C  =  Technology. 


SOURCE:  "Defense  Science  Board  Suatr  Study  on  Training  and  Training  Technology,” 
Final  Report,  November  1982. 
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APPENDIX  C 

SECRETARY  OF  DEFENSE  MEMORANDUM  APPROVING  DSB  RECOMMENDATIONS 


THE  SECRETARY  C'i  Di-TENSE 
WASHINGTON,  THE  DISTRICT  Of  COLUMBIA 

1 5  FIB  SO 


HEMORANC’wn  TO R  SECRET  AMES  OF  THE  MILITARY  DEPARTMENTS 

SUBJECT:  Defense  Science  Boerd  (DSB)  Sumer  Study  on  Training 
and  Training  Technology 


X  have  approved  the  recommendations  made  by  the  Defense 
Science  Board  to  improve  training  by  application  of  technology. 

The  report  and  approved  implementation  plan  are  attached.  X 
have  asked  the  Under  Secretary  of  Defense  for  Research  and 
Engineering  (USDRE)  to  take  the  lead  in  Initiating  the  actions 
called  for  in  the  plan.  X  have  also  asked  the  Assistant  Secretary 
of  Defense  for  Manpower,  Reserve  Affairs  and  Logistics  (ASD/MRA&L) 
to  assist  in  this  effort.  X  am  confident  that  the  implementation 
of  the  DSB  recommendations  will  improve  our  readiness  significantly. 

There  are  four  areas  that,  in  my  view,  demand  special  consider¬ 
ation  and  the  following  specific  guidance. 

1.  The  DoD  must  increase  its  funding  and  management  emphasis 
on  the  development  of  training  technology  and  the  use  of  this 
technology  to  address  training  problems.  Ke  must  take  advantage 
of  current  technology  and  press  the  research  of  emerging  techno¬ 
logies  to  develop  ways  to  make  training  more  efficient  and  effective 
To  support  this  effort,  it  will  be  necessary  to  develop  performance 
..ieasures  and  criteria  for  use  in  determining  performance  levels 

and  cost  effectiveness  of  alternative  training  methodologies. 

2.  Each  Service  should  accelerate  efforts  to  apply  technology 
to  meet  the  training  needs  of  the  Reserve  Components.  This 
investment  has  the  potential  for  very  high  payoff  and  merits  a 
high  priority.  X  ask  that  HRA&L  coordinate  this  effort. 

3.  There  is  an  urgent  requirement  to  upgrade  our  land, 
sea  and  air  ranges  to  make  them  as  compatible  as  possible  with 
the  needs  of  the  systems  being  tested  and  fielded.  Since  there 
are  clear  limits  to  real  estate  and  air  space,  the  capabilities 
of  existing  ranges  must  be  expanded  and  made  more  versatile. 

Each  Service  should  review  range  requirements  and  budget  funds 

to  upgrade  range  capabilities  to  include  realistic  threst  simula¬ 
tion  and  instrumentation  tor  total  operator/system  performance 
evaluation. 
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4.  X  major  continuing  weakness  of  the  overall  training 
system  is  the  abaenee  of  high-level  perspective  and  proponency 
for  training  technology.  Thla  ia  true,  with  aome  exeeptiona, 
in  both  OSD  and  the  Services.  Z  expect  that  each  of  you  will 
create  an  advocate  for  training  and  training  technology  within 

rour  own  secretariat.  General  Vessey  has  offered  to  take  the 
ead  in  establishing  an  OSD  Steering  Coaaittee  for  training  and 
training  technology.  Z  endorse  this  initiative  and  direct  the 
Coaaittee  to  consolidate  advocacy  consideration  within  OSD  on 
aatters  related  to  the  effective  exploitation  of  training  tech¬ 
nology. 


I  trust  that  you  are  as  iapressed  as  Z  an  with  the  quality 
of  the  DSB  study  and  will  support  all  of  its  reeoaaendations. 

Z  aa  requesting  the  OSD  Steering  Coaaittee  Include  in  its 
charter  a  process  to  review  the  lapleaentatlon  status  of  these 
reeoaaendations  as  well  as  other  actions  taken  to  strengthen 
training. 


Attachaent 
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APPENDIX  D 

NAVY  TRAINING  IMPROVEMENT  PROGRAM 


The  Navy  Training  Improvement  Program  (NAVTIP)  is  a  recent  initiative 
which  may  be  traced  to  Chief  of  Naval  Education  and  Training's  (CNET)  concerr 
about  the  cost-effectiveness  of  training,  the  Defense  Service  Board's  1982 
Summer  Study  in  Training  and  Training  Technology  (see  Appendix  B),  the  Under 
Secretary  of  Defense  for  Research  and  Engineering's  (USDRE)  memorandum  for  the 
Secretaries  of  the  Military  Departments  (4  April  1983)  assigning 
responsibilities  for  the  actions  recommended  by  the  Defense  Service  Board  a:-, 
approved  by  the  Secretary  of  Defense  (see  Appendix  C),  and  the  Chief  of  Nav*' 
Research  (CNR)  who  advocated  that  more  emphasis  be  put  on  transferring 
training  research  and  development  (R&D)  results  into  application.  The  NAVT J - 
Steering  Committee  function  is  part  of  CNET's  Training  R&D  Policy  Board. 

The  objective  of  NAVTIP  is  twofold: 

a.  Improving  training  through  the  immediate  application/implementation 
of  technology  (e.g.,  through  evaluation  of  R&D  product  demonstrations). 

b.  Modifying  and/or  augmenting  research  programs  to  improve  training 
technology  with  payoff  in  the  near  future. 

The  first  Steering  Committee  meeting  was  held  in  May  1983.  Work  group 
meetings  have  been  conducted  through  the  summer  to  develop  and  finalize  a  Plan 
of  Action  and  Milestones  composed  of  two  parts  responsive  to  the  two 
objectives.  The  first  part  includes  demonstrations/evaluations  in  the  areas 
of  computer-based  instruction,  automated  curriculum  development,  and  job 
performance  aids.  (The  first  two  coordinated  by  CNET,  the  third  by  the 
NAVTRAEQUIPCEN.)  The  second  part  is  a  five-year  R&D  plan,  coordinated  by  the 
Office  of  Naval  Research  (ONR).  For  the  whole  plan,  0P-01B7  (formerly  OP-115) 
serves  as  executive  secretary.  The  program  was  planned  to  take  effect  October 
1983,  but  has  been  deferred. 


155 


Technical  Information  Center 
Naval  Training  Equipment  Center 
Orlando,  FL  32813 
(2  copies) 

Defense  Technical 
Information  Center 
Cameron  Station 
Alexandria,  VA  22314 
(12  copies) 

Navy  Personnel  Research 
and  Development  Center 
Library 

San  Diego,  CA  92152 
(3  copies) 

Navy  Personnel  Research 
and  Development  Center 
Code  52  (Dr.  J.  HcLachlan) 

San  Diego,  CA  92152 
(6  copies) 

Logistics  Management  Institute 
6400  Goldsboro  Road 
Bethesda,  MD  20817 
(10  copies) 

Office  of  the  Deputy  Under 
Secretary  of  Defense 
OUSDRE  ( E&LS ) 

(CAPT  P.  R.  Chatelier) 

Pentagon,  Room  3D129 
Washington,  DC  20301 

Chief  of  Naval  Operations 

OP-01 

Washington,  DC  20350 

Chief  of  Naval  Operations 
0P-01B7  (Dr.  R.  Carroll) 
Washington,  DC  20350 

Chief  of  Naval  Operations 
OP- 11 IE  (LCDR  L.  Vincent) 
Washington,  DC  20350 


Chief  of  Naval  Operations 
OP-112 

Washington,  DC  20350 

Chief  of  Naval  Operations 
OP-29 

Washington,  DC  20350 

Chief  of  Naval  Operations 
OP-39 

Washington,  DC  20350 

Chief  of  Naval  Operations 
OP-59 

Washington,  DC  20350 

Chief  of  Naval  Operations 
0P-987H  (Dr.  R.  G.  Smith) 
Washington,  DC  20350 

Office  of  Naval  Research 
Code  270  (CAPT  P.  M.  Curran) 
800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval  Research 
Code  422  (Dr.  M.  Tolcott) 

800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval  Research 
Code  422TT  (Dr.  S.  Chipman) 
800  North  Quincy  Street 
Arlington,  VA  22217 


Office  of  Naval  Research 
ONT  0722  (Dr.  S.  Collyer) 

800  North  Quincy  Street 
Arlington,  VA  22217 

Office  of  Naval 

Acquisition  Support 
0NAS  041  (Mr.  M.  Aptner) 
Washington,  DC  20360 

Naval  Air  Systems  Command 
Code  5313X  (LCDR  T.  N.  Crosby) 
Washington,  DC  20361 


Naval  Air  Systems  Command 
Code  330J  (CDR  I.  Jones) 
Washington,  DC  20361 

Naval  Air  Systems  Command 
APC  205 -ON  (CDR  J.  N.  Owens) 
Washington,  DC  20361 

Naval  Air  Systems  Command 
APC  201 

Washington,  DC  20361 

Naval  Air  Systems  Command 
APC  202 

Washington,  DC  20361 

Naval  Air  Systems  Command 
APC  203 

Washington,  DC  20361 

Naval  Air  Systems  Command 
APC  204 

Washington,  DC  20361 

Naval  Air  Systems  Command 
PMA  244  (CAPT  Ridder) 
Washington,  DC  20361 

Naval  Sea  Systems  Command 
PUS  411G  (Mr.  N.  G.  Jenkins) 
Washington,  DC  20362 

Naval  Training  Equipment 
Center  Liaison  Office 
Code  L02  (Mr.  B.  Williams) 
CNET  Headquarters 
Naval  Air  Station 
Pensacola,  FL  32508-5100 

Chief  of  Naval  Education 
and  Training 
Code  00A  (Dr.  W.  Maloy) 

Naval  Air  Station 
Pensacola,  FL  32508 

Chief  of  Naval  Education 
and  Training 

Code  N-2  (CAPT  J.  B.  LeBlanc) 
Naval  Air  Station 
Pensacola,  FL  32508 


Chief  of  Naval  Education 
and  Training 

Code  N-211  (Dr.  D.  Davis) 

Naval  Air  Station 
Pensacola,  FL  32508 
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